In [ ]:
histogram_boxplot(data, 'CreditScore')
Businesses like banks which provide service have to worry about problem of 'Customer Churn' i.e. customers leaving and joining another service provider. It is important to understand which aspects of the service influence a customer's decision in this regard. Management can concentrate efforts on improvement of service, keeping in mind these priorities.
You as a Data scientist with the bank need to build a neural network based classifier that can determine whether a customer will leave the bank or not in the next 6 months.
CustomerId: Unique ID which is assigned to each customer
Surname: Last name of the customer
CreditScore: It defines the credit history of the customer.
Geography: A customer’s location
Gender: It defines the Gender of the customer
Age: Age of the customer
Tenure: Number of years for which the customer has been with the bank
NumOfProducts: refers to the number of products that a customer has purchased through the bank.
Balance: Account balance
HasCrCard: It is a categorical variable which decides whether the customer has credit card or not.
EstimatedSalary: Estimated salary
isActiveMember: Is is a categorical variable which decides whether the customer is active member of the bank or not ( Active member in the sense, using bank products regularly, making transactions etc )
Exited : whether or not the customer left the bank within six month. It can take two values 0=No ( Customer did not leave the bank ) 1=Yes ( Customer left the bank )
import pandas as pd # Library for data manipulation and analysis.
import numpy as np # Fundamental package for scientific computing.
import matplotlib.pyplot as plt # Plotting library for creating visualizations.
import seaborn as sns #For advanced visualizations.
import random # Library for generating random numbers.
from sklearn.model_selection import train_test_split # Function for splitting datasets for training and testing.
from sklearn.preprocessing import MinMaxScaler
from sklearn.metrics import confusion_matrix
from imblearn.over_sampling import SMOTE # Synthetic Minority Over-sampling Technique.
import time # Module for time-related operations.
import tensorflow as tf #An end-to-end open source machine learning platform
from tensorflow import keras # High-level neural networks API for deep learning.
from keras import backend # Abstraction layer for neural network backend engines.
from keras.models import Sequential # Model for building NN sequentially.
from keras.layers import Dense,Dropout # for creating fully connected neural network layers.
from tensorflow.keras.optimizers import SGD, Adam
# To suppress warnings
import warnings
warnings.filterwarnings("ignore")
# Set display options to avoid scientific notation
pd.set_option('display.float_format', '{:.2f}'.format)
# Set the option to display all columns
pd.set_option('display.max_columns', None)
# uncomment and run the following line if using Google Colab
from google.colab import drive
drive.mount('/content/drive')
Mounted at /content/drive
df = pd.read_csv("/content/drive/MyDrive/Personal/UT Austin/Project 4 - NN/Churn.csv")
data = df.copy()
RowNumber, CustomerID, and Surname won't be used in the analysis, so can remove them.
#Remove RowNumber, CustomerId and Surname columns from data
data.drop(['RowNumber','CustomerId','Surname'], axis=1, inplace=True)
data.head(5)
| CreditScore | Geography | Gender | Age | Tenure | Balance | NumOfProducts | HasCrCard | IsActiveMember | EstimatedSalary | Exited | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 619 | France | Female | 42 | 2 | 0.00 | 1 | 1 | 1 | 101348.88 | 1 |
| 1 | 608 | Spain | Female | 41 | 1 | 83807.86 | 1 | 0 | 1 | 112542.58 | 0 |
| 2 | 502 | France | Female | 42 | 8 | 159660.80 | 3 | 1 | 0 | 113931.57 | 1 |
| 3 | 699 | France | Female | 39 | 1 | 0.00 | 2 | 0 | 0 | 93826.63 | 0 |
| 4 | 850 | Spain | Female | 43 | 2 | 125510.82 | 1 | 1 | 1 | 79084.10 | 0 |
data.tail(5)
| CreditScore | Geography | Gender | Age | Tenure | Balance | NumOfProducts | HasCrCard | IsActiveMember | EstimatedSalary | Exited | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 9995 | 771 | France | Male | 39 | 5 | 0.00 | 2 | 1 | 0 | 96270.64 | 0 |
| 9996 | 516 | France | Male | 35 | 10 | 57369.61 | 1 | 1 | 1 | 101699.77 | 0 |
| 9997 | 709 | France | Female | 36 | 7 | 0.00 | 1 | 0 | 1 | 42085.58 | 1 |
| 9998 | 772 | Germany | Male | 42 | 3 | 75075.31 | 2 | 1 | 0 | 92888.52 | 1 |
| 9999 | 792 | France | Female | 28 | 4 | 130142.79 | 1 | 1 | 0 | 38190.78 | 0 |
data.shape
(10000, 11)
data.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 10000 entries, 0 to 9999 Data columns (total 11 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 CreditScore 10000 non-null int64 1 Geography 10000 non-null object 2 Gender 10000 non-null object 3 Age 10000 non-null int64 4 Tenure 10000 non-null int64 5 Balance 10000 non-null float64 6 NumOfProducts 10000 non-null int64 7 HasCrCard 10000 non-null int64 8 IsActiveMember 10000 non-null int64 9 EstimatedSalary 10000 non-null float64 10 Exited 10000 non-null int64 dtypes: float64(2), int64(7), object(2) memory usage: 859.5+ KB
#Get Summary Statistics for the numerical columns
data.describe().T
| count | mean | std | min | 25% | 50% | 75% | max | |
|---|---|---|---|---|---|---|---|---|
| CreditScore | 10000.0 | 650.528800 | 96.653299 | 350.00 | 584.00 | 652.000 | 718.0000 | 850.00 |
| Age | 10000.0 | 38.921800 | 10.487806 | 18.00 | 32.00 | 37.000 | 44.0000 | 92.00 |
| Tenure | 10000.0 | 5.012800 | 2.892174 | 0.00 | 3.00 | 5.000 | 7.0000 | 10.00 |
| Balance | 10000.0 | 76485.889288 | 62397.405202 | 0.00 | 0.00 | 97198.540 | 127644.2400 | 250898.09 |
| NumOfProducts | 10000.0 | 1.530200 | 0.581654 | 1.00 | 1.00 | 1.000 | 2.0000 | 4.00 |
| HasCrCard | 10000.0 | 0.705500 | 0.455840 | 0.00 | 0.00 | 1.000 | 1.0000 | 1.00 |
| IsActiveMember | 10000.0 | 0.515100 | 0.499797 | 0.00 | 0.00 | 1.000 | 1.0000 | 1.00 |
| EstimatedSalary | 10000.0 | 100090.239881 | 57510.492818 | 11.58 | 51002.11 | 100193.915 | 149388.2475 | 199992.48 |
| Exited | 10000.0 | 0.203700 | 0.402769 | 0.00 | 0.00 | 0.000 | 0.0000 | 1.00 |
# checking for null values
data.isnull().sum()
| 0 | |
|---|---|
| CreditScore | 0 |
| Geography | 0 |
| Gender | 0 |
| Age | 0 |
| Tenure | 0 |
| Balance | 0 |
| NumOfProducts | 0 |
| HasCrCard | 0 |
| IsActiveMember | 0 |
| EstimatedSalary | 0 |
| Exited | 0 |
No missing values!
#Checking for duplicate values
data.duplicated().sum()
0
No duplicate values!
# Get counts of each unique value
data['Exited'].value_counts()
| count | |
|---|---|
| Exited | |
| 0 | 7963 |
| 1 | 2037 |
# Get percentages of each unique value
data['Exited'].value_counts(normalize=True) * 100
| proportion | |
|---|---|
| Exited | |
| 0 | 79.63 |
| 1 | 20.37 |
Customers who left make up 20% of the total customers, so there is an imbalance the model may need to consider.
# function to plot a boxplot and a histogram along the same scale.
def histogram_boxplot(data, feature, figsize=(12, 7), kde=False, bins=None):
"""
Boxplot and histogram combined
data: dataframe
feature: dataframe column
figsize: size of figure (default (12,7))
kde: whether to the show density curve (default False)
bins: number of bins for histogram (default None)
"""
f2, (ax_box2, ax_hist2) = plt.subplots(
nrows=2, # Number of rows of the subplot grid= 2
sharex=True, # x-axis will be shared among all subplots
gridspec_kw={"height_ratios": (0.25, 0.75)},
figsize=figsize,
) # creating the 2 subplots
sns.boxplot(
data=data, x=feature, ax=ax_box2, showmeans=True, color="violet"
) # boxplot will be created and a triangle will indicate the mean value of the column
sns.histplot(
data=data, x=feature, kde=kde, ax=ax_hist2, bins=bins, palette="winter"
) if bins else sns.histplot(
data=data, x=feature, kde=kde, ax=ax_hist2
) # For histogram
ax_hist2.axvline(
data[feature].mean(), color="green", linestyle="--"
) # Add mean to the histogram
ax_hist2.axvline(
data[feature].median(), color="black", linestyle="-"
) # Add median to the histogram
# function to create labeled barplots
def labeled_barplot(data, feature, perc=False, n=None):
"""
Barplot with percentage at the top
data: dataframe
feature: dataframe column
perc: whether to display percentages instead of count (default is False)
n: displays the top n category levels (default is None, i.e., display all levels)
"""
total = len(data[feature]) # length of the column
count = data[feature].nunique()
if n is None:
plt.figure(figsize=(count + 1, 5))
else:
plt.figure(figsize=(n + 1, 5))
plt.xticks(rotation=90, fontsize=15)
ax = sns.countplot(
data=data,
x=feature,
palette="Paired",
order=data[feature].value_counts().index[:n].sort_values(),
)
for p in ax.patches:
if perc == True:
label = "{:.1f}%".format(
100 * p.get_height() / total
) # percentage of each class of the category
else:
label = p.get_height() # count of each level of the category
x = p.get_x() + p.get_width() / 2 # width of the plot
y = p.get_height() # height of the plot
ax.annotate(
label,
(x, y),
ha="center",
va="center",
size=12,
xytext=(0, 5),
textcoords="offset points",
) # annotate the percentage
plt.show() # show the plot
# function to plot stacked bar chart
def stacked_barplot(data, predictor, target):
"""
Print the category counts and plot a stacked bar chart
data: dataframe
predictor: independent variable
target: target variable
"""
count = data[predictor].nunique()
sorter = data[target].value_counts().index[-1]
tab1 = pd.crosstab(data[predictor], data[target], margins=True).sort_values(
by=sorter, ascending=False
)
print(tab1)
print("-" * 120)
tab = pd.crosstab(data[predictor], data[target], normalize="index").sort_values(
by=sorter, ascending=False
)
tab.plot(kind="bar", stacked=True, figsize=(count + 1, 5))
plt.legend(
loc="lower left", frameon=False,
)
plt.legend(loc="upper left", bbox_to_anchor=(1, 1))
plt.show()
### Function to plot distributions
def distribution_plot_wrt_target(data, predictor, target):
fig, axs = plt.subplots(2, 2, figsize=(12, 10))
target_uniq = data[target].unique()
axs[0, 0].set_title("Distribution of target for target=" + str(target_uniq[0]))
sns.histplot(
data=data[data[target] == target_uniq[0]],
x=predictor,
kde=True,
ax=axs[0, 0],
color="teal",
)
axs[0, 1].set_title("Distribution of target for target=" + str(target_uniq[1]))
sns.histplot(
data=data[data[target] == target_uniq[1]],
x=predictor,
kde=True,
ax=axs[0, 1],
color="orange",
)
axs[1, 0].set_title("Boxplot w.r.t target")
sns.boxplot(data=data, x=target, y=predictor, ax=axs[1, 0], palette="gist_rainbow")
axs[1, 1].set_title("Boxplot (without outliers) w.r.t target")
sns.boxplot(
data=data,
x=target,
y=predictor,
ax=axs[1, 1],
showfliers=False,
palette="gist_rainbow",
)
plt.tight_layout()
plt.show()
histogram_boxplot(data, 'CreditScore')
histogram_boxplot(data, 'Age')
histogram_boxplot(data, 'Tenure')
histogram_boxplot(data, 'Balance')
histogram_boxplot(data, 'NumOfProducts')
histogram_boxplot(data, 'EstimatedSalary')
labeled_barplot(data, 'Gender')
labeled_barplot(data, 'Geography')
labeled_barplot(data, 'HasCrCard')
labeled_barplot(data, 'IsActiveMember')
Let's look at the correlation matrix for all the numeric variables.
# Exclude specific columns from the correlation matrix
excluded_columns = ['Exited', 'IsActiveMember', 'HasCrCard']
filtered_data = data.drop(columns=excluded_columns)
# plotting the correlation heatmap
plt.figure(figsize=(12, 7))
sns.heatmap(filtered_data.corr(numeric_only=True), annot=True, fmt='0.2f', cmap='Spectral')
plt.show()
From the above we can see that there is almost no correlation between the numeric variables.
sns.pairplot(data,hue='Exited')
plt.show()
distribution_plot_wrt_target(data, 'CreditScore', 'Exited')
distribution_plot_wrt_target(data, 'Age', 'Exited')
distribution_plot_wrt_target(data, 'Tenure', 'Exited')
distribution_plot_wrt_target(data, 'Balance', 'Exited')
distribution_plot_wrt_target(data, 'NumOfProducts', 'Exited')
distribution_plot_wrt_target(data, 'EstimatedSalary', 'Exited')
stacked_barplot(data, 'Gender', 'Exited')
Exited 0 1 All Gender All 7963 2037 10000 Female 3404 1139 4543 Male 4559 898 5457 ------------------------------------------------------------------------------------------------------------------------
stacked_barplot(data, 'Geography', 'Exited')
Exited 0 1 All Geography All 7963 2037 10000 Germany 1695 814 2509 France 4204 810 5014 Spain 2064 413 2477 ------------------------------------------------------------------------------------------------------------------------
stacked_barplot(data, 'HasCrCard', 'Exited')
Exited 0 1 All HasCrCard All 7963 2037 10000 1 5631 1424 7055 0 2332 613 2945 ------------------------------------------------------------------------------------------------------------------------
stacked_barplot(data, 'IsActiveMember', 'Exited')
Exited 0 1 All IsActiveMember All 7963 2037 10000 0 3547 1302 4849 1 4416 735 5151 ------------------------------------------------------------------------------------------------------------------------
Lets convert the columns with an 'object' datatype into categorical variables, to reduce the space required to store the dataframe and improve processing efficieny.
for feature in data.columns: # Loop through all columns in the dataframe
if data[feature].dtype == 'object': # Only apply for columns with categorical strings
data[feature] = pd.Categorical(data[feature]) # Convert the column to a categorical type
# Confirm that column of type object have been converted to category
data.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 10000 entries, 0 to 9999 Data columns (total 11 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 CreditScore 10000 non-null int64 1 Geography 10000 non-null category 2 Gender 10000 non-null category 3 Age 10000 non-null int64 4 Tenure 10000 non-null int64 5 Balance 10000 non-null float64 6 NumOfProducts 10000 non-null int64 7 HasCrCard 10000 non-null int64 8 IsActiveMember 10000 non-null int64 9 EstimatedSalary 10000 non-null float64 10 Exited 10000 non-null int64 dtypes: category(2), float64(2), int64(7) memory usage: 723.0 KB
replaceStruct = {
"Gender": {"M": 0, "F":1},
"Geography": {"Germany": 0, "Spain":1 , "France": 2},
}
oneHotCols=["Gender","Geography"]
data=data.replace(replaceStruct)
data=pd.get_dummies(data, columns=oneHotCols, drop_first=True)
data.head(10)
| CreditScore | Age | Tenure | Balance | NumOfProducts | HasCrCard | IsActiveMember | EstimatedSalary | Exited | Gender_Male | Geography_0 | Geography_1 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 619 | 42 | 2 | 0.00 | 1 | 1 | 1 | 101348.88 | 1 | False | False | False |
| 1 | 608 | 41 | 1 | 83807.86 | 1 | 0 | 1 | 112542.58 | 0 | False | False | True |
| 2 | 502 | 42 | 8 | 159660.80 | 3 | 1 | 0 | 113931.57 | 1 | False | False | False |
| 3 | 699 | 39 | 1 | 0.00 | 2 | 0 | 0 | 93826.63 | 0 | False | False | False |
| 4 | 850 | 43 | 2 | 125510.82 | 1 | 1 | 1 | 79084.10 | 0 | False | False | True |
| 5 | 645 | 44 | 8 | 113755.78 | 2 | 1 | 0 | 149756.71 | 1 | True | False | True |
| 6 | 822 | 50 | 7 | 0.00 | 2 | 1 | 1 | 10062.80 | 0 | True | False | False |
| 7 | 376 | 29 | 4 | 115046.74 | 4 | 1 | 0 | 119346.88 | 1 | False | True | False |
| 8 | 501 | 44 | 4 | 142051.07 | 2 | 0 | 1 | 74940.50 | 0 | True | False | False |
| 9 | 684 | 27 | 2 | 134603.88 | 1 | 1 | 1 | 71725.73 | 0 | True | False | False |
data.info()
<class 'pandas.core.frame.DataFrame'> RangeIndex: 10000 entries, 0 to 9999 Data columns (total 12 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 CreditScore 10000 non-null int64 1 Age 10000 non-null int64 2 Tenure 10000 non-null int64 3 Balance 10000 non-null float64 4 NumOfProducts 10000 non-null int64 5 HasCrCard 10000 non-null int64 6 IsActiveMember 10000 non-null int64 7 EstimatedSalary 10000 non-null float64 8 Exited 10000 non-null int64 9 Gender_Male 10000 non-null bool 10 Geography_0 10000 non-null bool 11 Geography_1 10000 non-null bool dtypes: bool(3), float64(2), int64(7) memory usage: 732.5 KB
X = data.drop(columns=['Exited']) # Features (all except the target 'Exited')
y = data['Exited'] # Target
# Step 1: Split into 80% train+validation and 20% test
X_train_val, X_test, y_train_val, y_test = train_test_split(X, y, test_size=0.2, random_state=42, stratify=y)
# Step 2: Split the 80% train+validation into 75% train and 25% validation
X_train, X_val, y_train, y_val = train_test_split(X_train_val, y_train_val, test_size=0.25, random_state=42, stratify=y_train_val)
# Resulting splits:
# X_train, y_train -> 60% of the data (training set)
# X_val, y_val -> 20% of the data (validation set)
# X_test, y_test -> 20% of the data (test set)
# Print the shapes of the resulting datasets
print("Training set shape:", X_train.shape, y_train.shape)
print("Validation set shape:", X_val.shape, y_val.shape)
print("Test set shape:", X_test.shape, y_test.shape)
Training set shape: (6000, 11) (6000,) Validation set shape: (2000, 11) (2000,) Test set shape: (2000, 11) (2000,)
Now, we will perform scaling on the numerical variables separately for train, validation, and test sets.
scaler = MinMaxScaler()
# Here, we are passing all the features (numerical and categorical), that's okay as min-max scaler will not change values of categorical variables
X_train = scaler.fit_transform(X_train)
X_val = scaler.transform(X_val)
X_test = scaler.transform(X_test)
def plot(history, name, y_min=None, y_max=None):
"""
Function to plot loss/accuracy
history: an object which stores the metrics and losses.
name: can be one of Loss or Accuracy
y_min: Minimum value for y-axis (optional)
y_max: Maximum value for y-axis (optional)
"""
fig, ax = plt.subplots() # Creating a subplot with figure and axes.
plt.plot(history.history[name]) # Plotting the train accuracy or train loss
plt.plot(history.history['val_' + name]) # Plotting the validation accuracy or validation loss
plt.title('Model ' + name.capitalize()) # Defining the title of the plot.
plt.ylabel(name.capitalize()) # Capitalizing the first letter.
plt.xlabel('Epoch') # Defining the label for the x-axis.
fig.legend(['Train', 'Validation'], loc="upper right") # Defining the legend, loc controls the position of the legend.
# Setting y-axis limits if provided
if y_min is not None and y_max is not None:
plt.ylim(y_min, y_max)
plt.show()
We'll create a dataframe to store the results from all the models we build
#Defining the columns of the dataframe which are nothing but the hyper parameters and the metrics.
columns = ["model label","# hidden layers","# neurons - hidden layer","activation function - hidden layer","# epochs","batch size","train loss","validation loss","train accuracy","validation accuracy","train recall","validation recall","time (secs)"]
#Creating a pandas dataframe.
results = pd.DataFrame(columns=columns)
# Function to plot confusion matrix with percentages and metrics
def plot_confusion_matrix(model, predictors, target):
"""
To plot the confusion_matrix with percentages and additional metrics
model: classifier
predictors: independent variables
target: dependent variable
"""
from sklearn.metrics import confusion_matrix
import seaborn as sns
# Predict the target values using the provided model and predictors
y_pred = (model.predict(predictors, verbose=0) > 0.5).astype(int)
# Compute the confusion matrix comparing the true target values with the predicted values
cm = confusion_matrix(target, y_pred)
# Create labels for each cell in the confusion matrix with both count and percentage
labels = np.asarray(
[
["{0:0.0f}".format(item) + "\n{0:.2%}".format(item / cm.sum())]
for item in cm.flatten()
]
).reshape(2, 2) # reshaping to a matrix
# Set the figure size for the plot
plt.figure(figsize=(8, 6))
# Plot the confusion matrix as a heatmap with the labels
sns.heatmap(cm, annot=labels, fmt="", cmap='Blues')
# Add a label to the y-axis
plt.ylabel("True label")
# Add a label to the x-axis
plt.xlabel("Predicted label")
# Compute and display additional metrics
accuracy = np.trace(cm) / float(np.sum(cm))
if len(cm) == 2:
precision = cm[1, 1] / sum(cm[:, 1])
recall = cm[1, 1] / sum(cm[1, :])
f1 = 2 * precision * recall / (precision + recall)
stats_text = "\n\nAccuracy={:0.2f}\nPrecision={:0.2f}\nRecall={:0.2f}\nF1 Score={:0.2f}".format(
accuracy, precision, recall, f1
)
else:
stats_text = "\n\nAccuracy={:0.2f}".format(accuracy)
# Set the title and additional metrics
plt.xlabel("Predicted label" + stats_text)
plt.title("Confusion Matrix")
plt.show()
Since we are trying to identify those customer who are at risk of churning and we want to not miss any of the potential ones, therefore, Recall is good metric to use. However, since Exited is imbalanced (80/20) Recall can pose problems as it doesn't account for false-positives. This is where use of SMOTE will come in handy to deal with the class imbalance.
Set the seed for random number generators in NumPy, Python, and TensorFlow to be able to reproduce the same results everytime we run the code.
# Fixing the seed for random number generators
np.random.seed(42)
random.seed(42)
tf.random.set_seed(42)
smote = SMOTE(random_state=42)
We'll start out with a baseline model having the following configuration:
# clears the current Keras session, resetting all layers and models previously created, freeing up memory and resources.
tf.keras.backend.clear_session()
#Initializing the neural network
model_0 = Sequential()
model_0.add(Dense(64, activation="relu",input_dim = X_train.shape[1])) #Hidden & Input Layer
model_0.add(Dense(32,activation="relu")) #Hidden Layer
model_0.add(Dense(1,activation = "sigmoid")) #Output Layer
#Printing the summary.
model_0.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩ │ dense (Dense) │ (None, 64) │ 768 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_1 (Dense) │ (None, 32) │ 2,080 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_2 (Dense) │ (None, 1) │ 33 │ └──────────────────────────────────────┴─────────────────────────────┴─────────────────┘
Total params: 2,881 (11.25 KB)
Trainable params: 2,881 (11.25 KB)
Non-trainable params: 0 (0.00 B)
optimizer = keras.optimizers.SGD(learning_rate=0.005, momentum=0.9) # defining SGD as the optimizer to be used
model_0.compile(optimizer=optimizer, loss='binary_crossentropy', metrics=['accuracy', tf.keras.metrics.Recall(name='recall')])
#batch_size = X_train.shape[0]
batch_size = 64
epochs = 100
start = time.time()
history = model_0.fit(X_train, y_train, validation_data=(X_val,y_val) , batch_size=batch_size, epochs=epochs)
end=time.time()
Epoch 1/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 2s 6ms/step - accuracy: 0.7285 - loss: 0.5957 - recall: 0.1896 - val_accuracy: 0.7965 - val_loss: 0.4898 - val_recall: 0.0000e+00 Epoch 2/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7979 - loss: 0.4839 - recall: 0.0000e+00 - val_accuracy: 0.7965 - val_loss: 0.4825 - val_recall: 0.0000e+00 Epoch 3/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7979 - loss: 0.4763 - recall: 0.0000e+00 - val_accuracy: 0.7965 - val_loss: 0.4770 - val_recall: 0.0000e+00 Epoch 4/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7979 - loss: 0.4704 - recall: 0.0000e+00 - val_accuracy: 0.7965 - val_loss: 0.4732 - val_recall: 0.0000e+00 Epoch 5/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7979 - loss: 0.4659 - recall: 0.0000e+00 - val_accuracy: 0.7965 - val_loss: 0.4699 - val_recall: 0.0000e+00 Epoch 6/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7979 - loss: 0.4620 - recall: 0.0000e+00 - val_accuracy: 0.7965 - val_loss: 0.4669 - val_recall: 0.0000e+00 Epoch 7/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 4ms/step - accuracy: 0.7979 - loss: 0.4585 - recall: 0.0000e+00 - val_accuracy: 0.7965 - val_loss: 0.4638 - val_recall: 0.0000e+00 Epoch 8/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7981 - loss: 0.4553 - recall: 0.0016 - val_accuracy: 0.7960 - val_loss: 0.4607 - val_recall: 0.0000e+00 Epoch 9/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7998 - loss: 0.4520 - recall: 0.0124 - val_accuracy: 0.7995 - val_loss: 0.4576 - val_recall: 0.0197 Epoch 10/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8042 - loss: 0.4489 - recall: 0.0377 - val_accuracy: 0.8005 - val_loss: 0.4542 - val_recall: 0.0319 Epoch 11/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 4ms/step - accuracy: 0.8039 - loss: 0.4457 - recall: 0.0488 - val_accuracy: 0.8010 - val_loss: 0.4507 - val_recall: 0.0467 Epoch 12/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8056 - loss: 0.4426 - recall: 0.0663 - val_accuracy: 0.8020 - val_loss: 0.4474 - val_recall: 0.0639 Epoch 13/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8083 - loss: 0.4395 - recall: 0.0858 - val_accuracy: 0.8030 - val_loss: 0.4441 - val_recall: 0.0811 Epoch 14/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8119 - loss: 0.4364 - recall: 0.1144 - val_accuracy: 0.8065 - val_loss: 0.4411 - val_recall: 0.0983 Epoch 15/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8158 - loss: 0.4337 - recall: 0.1375 - val_accuracy: 0.8075 - val_loss: 0.4383 - val_recall: 0.1106 Epoch 16/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8191 - loss: 0.4312 - recall: 0.1618 - val_accuracy: 0.8110 - val_loss: 0.4359 - val_recall: 0.1278 Epoch 17/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8176 - loss: 0.4290 - recall: 0.1696 - val_accuracy: 0.8115 - val_loss: 0.4336 - val_recall: 0.1499 Epoch 18/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8169 - loss: 0.4271 - recall: 0.1792 - val_accuracy: 0.8120 - val_loss: 0.4317 - val_recall: 0.1622 Epoch 19/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.8169 - loss: 0.4254 - recall: 0.1950 - val_accuracy: 0.8140 - val_loss: 0.4302 - val_recall: 0.1744 Epoch 20/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.8181 - loss: 0.4239 - recall: 0.2063 - val_accuracy: 0.8140 - val_loss: 0.4288 - val_recall: 0.1818 Epoch 21/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8185 - loss: 0.4225 - recall: 0.2134 - val_accuracy: 0.8140 - val_loss: 0.4277 - val_recall: 0.1867 Epoch 22/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8204 - loss: 0.4212 - recall: 0.2242 - val_accuracy: 0.8165 - val_loss: 0.4265 - val_recall: 0.2015 Epoch 23/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8197 - loss: 0.4201 - recall: 0.2304 - val_accuracy: 0.8180 - val_loss: 0.4254 - val_recall: 0.2088 Epoch 24/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.8191 - loss: 0.4188 - recall: 0.2326 - val_accuracy: 0.8195 - val_loss: 0.4243 - val_recall: 0.2211 Epoch 25/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8187 - loss: 0.4175 - recall: 0.2343 - val_accuracy: 0.8195 - val_loss: 0.4232 - val_recall: 0.2236 Epoch 26/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8191 - loss: 0.4160 - recall: 0.2396 - val_accuracy: 0.8205 - val_loss: 0.4220 - val_recall: 0.2285 Epoch 27/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8192 - loss: 0.4145 - recall: 0.2419 - val_accuracy: 0.8210 - val_loss: 0.4208 - val_recall: 0.2334 Epoch 28/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8207 - loss: 0.4130 - recall: 0.2485 - val_accuracy: 0.8215 - val_loss: 0.4196 - val_recall: 0.2359 Epoch 29/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8217 - loss: 0.4114 - recall: 0.2551 - val_accuracy: 0.8215 - val_loss: 0.4182 - val_recall: 0.2359 Epoch 30/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8240 - loss: 0.4096 - recall: 0.2646 - val_accuracy: 0.8225 - val_loss: 0.4164 - val_recall: 0.2432 Epoch 31/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8242 - loss: 0.4075 - recall: 0.2682 - val_accuracy: 0.8230 - val_loss: 0.4146 - val_recall: 0.2482 Epoch 32/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8259 - loss: 0.4053 - recall: 0.2740 - val_accuracy: 0.8250 - val_loss: 0.4128 - val_recall: 0.2629 Epoch 33/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8264 - loss: 0.4030 - recall: 0.2785 - val_accuracy: 0.8255 - val_loss: 0.4109 - val_recall: 0.2727 Epoch 34/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8285 - loss: 0.4007 - recall: 0.2854 - val_accuracy: 0.8265 - val_loss: 0.4086 - val_recall: 0.2826 Epoch 35/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8292 - loss: 0.3980 - recall: 0.2869 - val_accuracy: 0.8285 - val_loss: 0.4061 - val_recall: 0.2924 Epoch 36/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8318 - loss: 0.3952 - recall: 0.2991 - val_accuracy: 0.8295 - val_loss: 0.4034 - val_recall: 0.2948 Epoch 37/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8346 - loss: 0.3924 - recall: 0.3037 - val_accuracy: 0.8310 - val_loss: 0.4005 - val_recall: 0.3022 Epoch 38/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8354 - loss: 0.3895 - recall: 0.3087 - val_accuracy: 0.8305 - val_loss: 0.3975 - val_recall: 0.3047 Epoch 39/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8379 - loss: 0.3863 - recall: 0.3192 - val_accuracy: 0.8320 - val_loss: 0.3944 - val_recall: 0.3145 Epoch 40/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.8396 - loss: 0.3830 - recall: 0.3310 - val_accuracy: 0.8375 - val_loss: 0.3910 - val_recall: 0.3415 Epoch 41/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8434 - loss: 0.3797 - recall: 0.3433 - val_accuracy: 0.8385 - val_loss: 0.3876 - val_recall: 0.3464 Epoch 42/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8444 - loss: 0.3765 - recall: 0.3473 - val_accuracy: 0.8405 - val_loss: 0.3844 - val_recall: 0.3612 Epoch 43/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8457 - loss: 0.3733 - recall: 0.3592 - val_accuracy: 0.8415 - val_loss: 0.3811 - val_recall: 0.3710 Epoch 44/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8460 - loss: 0.3702 - recall: 0.3647 - val_accuracy: 0.8450 - val_loss: 0.3780 - val_recall: 0.3907 Epoch 45/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8472 - loss: 0.3672 - recall: 0.3693 - val_accuracy: 0.8460 - val_loss: 0.3749 - val_recall: 0.3931 Epoch 46/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 4ms/step - accuracy: 0.8503 - loss: 0.3643 - recall: 0.3736 - val_accuracy: 0.8475 - val_loss: 0.3723 - val_recall: 0.3931 Epoch 47/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8516 - loss: 0.3616 - recall: 0.3797 - val_accuracy: 0.8500 - val_loss: 0.3696 - val_recall: 0.4005 Epoch 48/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8524 - loss: 0.3592 - recall: 0.3865 - val_accuracy: 0.8510 - val_loss: 0.3674 - val_recall: 0.4029 Epoch 49/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8525 - loss: 0.3570 - recall: 0.3927 - val_accuracy: 0.8510 - val_loss: 0.3654 - val_recall: 0.4029 Epoch 50/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.8534 - loss: 0.3550 - recall: 0.3982 - val_accuracy: 0.8510 - val_loss: 0.3636 - val_recall: 0.4054 Epoch 51/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8541 - loss: 0.3532 - recall: 0.4027 - val_accuracy: 0.8530 - val_loss: 0.3620 - val_recall: 0.4152 Epoch 52/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8538 - loss: 0.3516 - recall: 0.4034 - val_accuracy: 0.8520 - val_loss: 0.3607 - val_recall: 0.4103 Epoch 53/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8549 - loss: 0.3502 - recall: 0.4108 - val_accuracy: 0.8520 - val_loss: 0.3596 - val_recall: 0.4103 Epoch 54/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8559 - loss: 0.3490 - recall: 0.4198 - val_accuracy: 0.8540 - val_loss: 0.3586 - val_recall: 0.4226 Epoch 55/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8562 - loss: 0.3478 - recall: 0.4229 - val_accuracy: 0.8545 - val_loss: 0.3577 - val_recall: 0.4251 Epoch 56/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8563 - loss: 0.3468 - recall: 0.4272 - val_accuracy: 0.8560 - val_loss: 0.3569 - val_recall: 0.4300 Epoch 57/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8557 - loss: 0.3459 - recall: 0.4267 - val_accuracy: 0.8555 - val_loss: 0.3563 - val_recall: 0.4300 Epoch 58/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.8563 - loss: 0.3451 - recall: 0.4309 - val_accuracy: 0.8550 - val_loss: 0.3557 - val_recall: 0.4275 Epoch 59/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8555 - loss: 0.3444 - recall: 0.4314 - val_accuracy: 0.8550 - val_loss: 0.3553 - val_recall: 0.4349 Epoch 60/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.8564 - loss: 0.3436 - recall: 0.4319 - val_accuracy: 0.8540 - val_loss: 0.3549 - val_recall: 0.4300 Epoch 61/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8563 - loss: 0.3430 - recall: 0.4299 - val_accuracy: 0.8525 - val_loss: 0.3545 - val_recall: 0.4251 Epoch 62/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8568 - loss: 0.3425 - recall: 0.4322 - val_accuracy: 0.8535 - val_loss: 0.3542 - val_recall: 0.4324 Epoch 63/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.8570 - loss: 0.3421 - recall: 0.4320 - val_accuracy: 0.8535 - val_loss: 0.3537 - val_recall: 0.4324 Epoch 64/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8576 - loss: 0.3416 - recall: 0.4325 - val_accuracy: 0.8540 - val_loss: 0.3535 - val_recall: 0.4349 Epoch 65/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8571 - loss: 0.3410 - recall: 0.4327 - val_accuracy: 0.8550 - val_loss: 0.3532 - val_recall: 0.4373 Epoch 66/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8570 - loss: 0.3405 - recall: 0.4331 - val_accuracy: 0.8555 - val_loss: 0.3529 - val_recall: 0.4373 Epoch 67/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8583 - loss: 0.3401 - recall: 0.4332 - val_accuracy: 0.8550 - val_loss: 0.3527 - val_recall: 0.4349 Epoch 68/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8577 - loss: 0.3398 - recall: 0.4381 - val_accuracy: 0.8555 - val_loss: 0.3527 - val_recall: 0.4324 Epoch 69/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8586 - loss: 0.3394 - recall: 0.4417 - val_accuracy: 0.8550 - val_loss: 0.3525 - val_recall: 0.4324 Epoch 70/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8595 - loss: 0.3390 - recall: 0.4465 - val_accuracy: 0.8550 - val_loss: 0.3523 - val_recall: 0.4324 Epoch 71/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8592 - loss: 0.3388 - recall: 0.4459 - val_accuracy: 0.8540 - val_loss: 0.3522 - val_recall: 0.4251 Epoch 72/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8595 - loss: 0.3384 - recall: 0.4472 - val_accuracy: 0.8545 - val_loss: 0.3521 - val_recall: 0.4275 Epoch 73/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8594 - loss: 0.3380 - recall: 0.4482 - val_accuracy: 0.8545 - val_loss: 0.3519 - val_recall: 0.4300 Epoch 74/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8591 - loss: 0.3379 - recall: 0.4486 - val_accuracy: 0.8540 - val_loss: 0.3518 - val_recall: 0.4300 Epoch 75/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8588 - loss: 0.3377 - recall: 0.4473 - val_accuracy: 0.8555 - val_loss: 0.3515 - val_recall: 0.4349 Epoch 76/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 6ms/step - accuracy: 0.8599 - loss: 0.3374 - recall: 0.4516 - val_accuracy: 0.8560 - val_loss: 0.3513 - val_recall: 0.4373 Epoch 77/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 4ms/step - accuracy: 0.8603 - loss: 0.3370 - recall: 0.4541 - val_accuracy: 0.8565 - val_loss: 0.3510 - val_recall: 0.4373 Epoch 78/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8605 - loss: 0.3370 - recall: 0.4564 - val_accuracy: 0.8555 - val_loss: 0.3510 - val_recall: 0.4373 Epoch 79/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8602 - loss: 0.3368 - recall: 0.4545 - val_accuracy: 0.8550 - val_loss: 0.3509 - val_recall: 0.4324 Epoch 80/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8613 - loss: 0.3364 - recall: 0.4580 - val_accuracy: 0.8550 - val_loss: 0.3510 - val_recall: 0.4324 Epoch 81/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 4ms/step - accuracy: 0.8620 - loss: 0.3361 - recall: 0.4593 - val_accuracy: 0.8540 - val_loss: 0.3509 - val_recall: 0.4324 Epoch 82/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8612 - loss: 0.3360 - recall: 0.4593 - val_accuracy: 0.8540 - val_loss: 0.3509 - val_recall: 0.4275 Epoch 83/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8618 - loss: 0.3357 - recall: 0.4606 - val_accuracy: 0.8555 - val_loss: 0.3507 - val_recall: 0.4324 Epoch 84/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8619 - loss: 0.3355 - recall: 0.4610 - val_accuracy: 0.8550 - val_loss: 0.3507 - val_recall: 0.4324 Epoch 85/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8624 - loss: 0.3352 - recall: 0.4623 - val_accuracy: 0.8545 - val_loss: 0.3506 - val_recall: 0.4324 Epoch 86/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8630 - loss: 0.3349 - recall: 0.4633 - val_accuracy: 0.8560 - val_loss: 0.3507 - val_recall: 0.4324 Epoch 87/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.8628 - loss: 0.3348 - recall: 0.4625 - val_accuracy: 0.8545 - val_loss: 0.3504 - val_recall: 0.4324 Epoch 88/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8629 - loss: 0.3346 - recall: 0.4610 - val_accuracy: 0.8550 - val_loss: 0.3506 - val_recall: 0.4324 Epoch 89/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8635 - loss: 0.3344 - recall: 0.4654 - val_accuracy: 0.8545 - val_loss: 0.3505 - val_recall: 0.4324 Epoch 90/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8630 - loss: 0.3341 - recall: 0.4628 - val_accuracy: 0.8560 - val_loss: 0.3505 - val_recall: 0.4300 Epoch 91/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8628 - loss: 0.3338 - recall: 0.4642 - val_accuracy: 0.8560 - val_loss: 0.3506 - val_recall: 0.4300 Epoch 92/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8630 - loss: 0.3337 - recall: 0.4651 - val_accuracy: 0.8560 - val_loss: 0.3504 - val_recall: 0.4300 Epoch 93/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8628 - loss: 0.3335 - recall: 0.4645 - val_accuracy: 0.8565 - val_loss: 0.3505 - val_recall: 0.4300 Epoch 94/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8628 - loss: 0.3334 - recall: 0.4638 - val_accuracy: 0.8570 - val_loss: 0.3504 - val_recall: 0.4324 Epoch 95/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8627 - loss: 0.3331 - recall: 0.4640 - val_accuracy: 0.8580 - val_loss: 0.3503 - val_recall: 0.4349 Epoch 96/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8622 - loss: 0.3329 - recall: 0.4652 - val_accuracy: 0.8575 - val_loss: 0.3503 - val_recall: 0.4324 Epoch 97/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8628 - loss: 0.3327 - recall: 0.4655 - val_accuracy: 0.8575 - val_loss: 0.3503 - val_recall: 0.4300 Epoch 98/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8632 - loss: 0.3324 - recall: 0.4674 - val_accuracy: 0.8565 - val_loss: 0.3503 - val_recall: 0.4324 Epoch 99/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8632 - loss: 0.3323 - recall: 0.4702 - val_accuracy: 0.8570 - val_loss: 0.3503 - val_recall: 0.4349 Epoch 100/100 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8628 - loss: 0.3321 - recall: 0.4686 - val_accuracy: 0.8570 - val_loss: 0.3504 - val_recall: 0.4349
print("Time taken in seconds ",end-start)
Time taken in seconds 42.23354721069336
plot(history,'loss')
plot(history,'accuracy')
plot(history,'recall')
#print(history.history.keys())
results.loc[0] = ['NN with SGD',2,'[64,32]','relu',epochs,batch_size,history.history["loss"][-1],history.history["val_loss"][-1],history.history["accuracy"][-1],history.history["val_accuracy"][-1],history.history["recall"][-1],history.history["val_recall"][-1],round(end-start,2)]
results
| model label | # hidden layers | # neurons - hidden layer | activation function - hidden layer | # epochs | batch size | train loss | validation loss | train accuracy | validation accuracy | train recall | validation recall | time (secs) | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | NN with SGD | 2 | [64,32] | relu | 100 | 64 | 0.33 | 0.35 | 0.86 | 0.86 | 0.47 | 0.43 | 42.23 |
Commentary: This model has a good balance between training and validation, suggesting limited overfitting, however, the recall score is not very good. I also tried using tanh in the second hidden layer, however, the peformance was not significantly different, so stayed with using relu in both hidden layers.
# clears the current Keras session, resetting all layers and models previously created, freeing up memory and resources.
tf.keras.backend.clear_session()
#Initializing the neural network
model_1 = Sequential()
model_1.add(Dense(64, activation="relu",input_dim = X_train.shape[1])) #Hidden & Input Layer
model_1.add(Dense(32,activation="relu")) #Hidden Layer
model_1.add(Dense(1,activation = "sigmoid")) #Output Layer
#Printing the summary.
model_1.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩ │ dense (Dense) │ (None, 64) │ 768 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_1 (Dense) │ (None, 32) │ 2,080 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_2 (Dense) │ (None, 1) │ 33 │ └──────────────────────────────────────┴─────────────────────────────┴─────────────────┘
Total params: 2,881 (11.25 KB)
Trainable params: 2,881 (11.25 KB)
Non-trainable params: 0 (0.00 B)
optimizer = tf.keras.optimizers.Adam(learning_rate=0.005) # defining Adam as the optimizer to be used
model_1.compile(optimizer=optimizer, loss='binary_crossentropy', metrics=['accuracy', tf.keras.metrics.Recall(name='recall')])
batch_size = 64
epochs = 40
start = time.time()
history = model_1.fit(X_train, y_train, validation_data=(X_val,y_val) , batch_size=batch_size, epochs=epochs)
end=time.time()
Epoch 1/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 2s 9ms/step - accuracy: 0.7160 - loss: 0.5444 - recall: 0.1123 - val_accuracy: 0.8015 - val_loss: 0.4504 - val_recall: 0.0442 Epoch 2/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8127 - loss: 0.4348 - recall: 0.1266 - val_accuracy: 0.8230 - val_loss: 0.4228 - val_recall: 0.3464 Epoch 3/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 6ms/step - accuracy: 0.8228 - loss: 0.4121 - recall: 0.2707 - val_accuracy: 0.8305 - val_loss: 0.3986 - val_recall: 0.4128 Epoch 4/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8416 - loss: 0.3870 - recall: 0.3750 - val_accuracy: 0.8375 - val_loss: 0.3766 - val_recall: 0.4423 Epoch 5/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 6ms/step - accuracy: 0.8460 - loss: 0.3718 - recall: 0.4097 - val_accuracy: 0.8440 - val_loss: 0.3656 - val_recall: 0.4423 Epoch 6/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8510 - loss: 0.3589 - recall: 0.4234 - val_accuracy: 0.8480 - val_loss: 0.3607 - val_recall: 0.4275 Epoch 7/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8560 - loss: 0.3512 - recall: 0.4387 - val_accuracy: 0.8505 - val_loss: 0.3585 - val_recall: 0.4423 Epoch 8/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8556 - loss: 0.3459 - recall: 0.4437 - val_accuracy: 0.8520 - val_loss: 0.3568 - val_recall: 0.4349 Epoch 9/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8566 - loss: 0.3411 - recall: 0.4554 - val_accuracy: 0.8520 - val_loss: 0.3561 - val_recall: 0.4201 Epoch 10/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8581 - loss: 0.3380 - recall: 0.4572 - val_accuracy: 0.8505 - val_loss: 0.3574 - val_recall: 0.4054 Epoch 11/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8580 - loss: 0.3357 - recall: 0.4631 - val_accuracy: 0.8500 - val_loss: 0.3578 - val_recall: 0.3956 Epoch 12/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8609 - loss: 0.3346 - recall: 0.4785 - val_accuracy: 0.8500 - val_loss: 0.3569 - val_recall: 0.4054 Epoch 13/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8620 - loss: 0.3321 - recall: 0.4814 - val_accuracy: 0.8530 - val_loss: 0.3562 - val_recall: 0.4029 Epoch 14/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8630 - loss: 0.3317 - recall: 0.4850 - val_accuracy: 0.8510 - val_loss: 0.3579 - val_recall: 0.4005 Epoch 15/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8637 - loss: 0.3308 - recall: 0.4866 - val_accuracy: 0.8510 - val_loss: 0.3579 - val_recall: 0.4029 Epoch 16/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8617 - loss: 0.3304 - recall: 0.4749 - val_accuracy: 0.8530 - val_loss: 0.3573 - val_recall: 0.4054 Epoch 17/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8620 - loss: 0.3291 - recall: 0.4767 - val_accuracy: 0.8495 - val_loss: 0.3589 - val_recall: 0.4029 Epoch 18/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8619 - loss: 0.3287 - recall: 0.4728 - val_accuracy: 0.8520 - val_loss: 0.3580 - val_recall: 0.4177 Epoch 19/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8604 - loss: 0.3277 - recall: 0.4717 - val_accuracy: 0.8530 - val_loss: 0.3601 - val_recall: 0.4152 Epoch 20/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8599 - loss: 0.3268 - recall: 0.4683 - val_accuracy: 0.8515 - val_loss: 0.3603 - val_recall: 0.4251 Epoch 21/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8603 - loss: 0.3261 - recall: 0.4708 - val_accuracy: 0.8520 - val_loss: 0.3600 - val_recall: 0.4300 Epoch 22/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8599 - loss: 0.3253 - recall: 0.4697 - val_accuracy: 0.8545 - val_loss: 0.3614 - val_recall: 0.4373 Epoch 23/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8591 - loss: 0.3245 - recall: 0.4706 - val_accuracy: 0.8530 - val_loss: 0.3619 - val_recall: 0.4447 Epoch 24/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8612 - loss: 0.3239 - recall: 0.4780 - val_accuracy: 0.8525 - val_loss: 0.3623 - val_recall: 0.4496 Epoch 25/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8609 - loss: 0.3236 - recall: 0.4752 - val_accuracy: 0.8520 - val_loss: 0.3625 - val_recall: 0.4521 Epoch 26/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8611 - loss: 0.3228 - recall: 0.4790 - val_accuracy: 0.8540 - val_loss: 0.3620 - val_recall: 0.4545 Epoch 27/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8615 - loss: 0.3217 - recall: 0.4757 - val_accuracy: 0.8540 - val_loss: 0.3632 - val_recall: 0.4521 Epoch 28/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8613 - loss: 0.3221 - recall: 0.4733 - val_accuracy: 0.8550 - val_loss: 0.3647 - val_recall: 0.4693 Epoch 29/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8625 - loss: 0.3209 - recall: 0.4811 - val_accuracy: 0.8545 - val_loss: 0.3664 - val_recall: 0.4619 Epoch 30/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 4ms/step - accuracy: 0.8621 - loss: 0.3207 - recall: 0.4800 - val_accuracy: 0.8530 - val_loss: 0.3665 - val_recall: 0.4619 Epoch 31/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8641 - loss: 0.3202 - recall: 0.4847 - val_accuracy: 0.8535 - val_loss: 0.3660 - val_recall: 0.4595 Epoch 32/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8634 - loss: 0.3193 - recall: 0.4811 - val_accuracy: 0.8550 - val_loss: 0.3659 - val_recall: 0.4570 Epoch 33/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8640 - loss: 0.3185 - recall: 0.4847 - val_accuracy: 0.8550 - val_loss: 0.3654 - val_recall: 0.4595 Epoch 34/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8644 - loss: 0.3182 - recall: 0.4897 - val_accuracy: 0.8545 - val_loss: 0.3646 - val_recall: 0.4619 Epoch 35/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8652 - loss: 0.3178 - recall: 0.4928 - val_accuracy: 0.8540 - val_loss: 0.3641 - val_recall: 0.4619 Epoch 36/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 6ms/step - accuracy: 0.8662 - loss: 0.3171 - recall: 0.4968 - val_accuracy: 0.8525 - val_loss: 0.3683 - val_recall: 0.4570 Epoch 37/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8654 - loss: 0.3167 - recall: 0.4904 - val_accuracy: 0.8540 - val_loss: 0.3688 - val_recall: 0.4570 Epoch 38/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8655 - loss: 0.3165 - recall: 0.4919 - val_accuracy: 0.8505 - val_loss: 0.3687 - val_recall: 0.4472 Epoch 39/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8663 - loss: 0.3165 - recall: 0.4927 - val_accuracy: 0.8525 - val_loss: 0.3684 - val_recall: 0.4595 Epoch 40/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8674 - loss: 0.3157 - recall: 0.4922 - val_accuracy: 0.8495 - val_loss: 0.3694 - val_recall: 0.4496
print("Time taken in seconds ",end-start)
Time taken in seconds 20.51025629043579
plot(history,'loss')
plot(history,'accuracy')
plot(history,'recall')
results.loc[1] = ['NN with Adam',2,'[64,32]','relu',epochs,batch_size,history.history["loss"][-1],history.history["val_loss"][-1],history.history["accuracy"][-1],history.history["val_accuracy"][-1],history.history["recall"][-1],history.history["val_recall"][-1],round(end-start,2)]
results = results.sort_index()
results
| model label | # hidden layers | # neurons - hidden layer | activation function - hidden layer | # epochs | batch size | train loss | validation loss | train accuracy | validation accuracy | train recall | validation recall | time (secs) | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | NN with SGD | 2 | [64,32] | relu | 100 | 64 | 0.33 | 0.35 | 0.86 | 0.86 | 0.47 | 0.43 | 42.23 |
| 1 | NN with Adam | 2 | [64,32] | relu | 40 | 64 | 0.31 | 0.37 | 0.87 | 0.85 | 0.51 | 0.45 | 20.51 |
Commentary: This model achieved slightly better recall scores, while also not overfitting. The model also required fewer Epochs to converge, saving some compute time. So an improvement, but recall scores are still not where they need to be.
# clears the current Keras session, resetting all layers and models previously created, freeing up memory and resources.
tf.keras.backend.clear_session()
#Initializing the neural network
model_2 = Sequential()
model_2.add(Dense(64, activation="relu",input_dim = X_train.shape[1])) #Hidden & Input Layer
model_2.add(Dropout(0.3)) #Dropout Layer
model_2.add(Dense(32,activation="relu")) #Hidden Layer
model_2.add(Dropout(0.3)) #Dropout Layer
model_2.add(Dense(1,activation = "sigmoid")) #Output Layer
#Printing the summary.
model_2.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩ │ dense (Dense) │ (None, 64) │ 768 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dropout (Dropout) │ (None, 64) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_1 (Dense) │ (None, 32) │ 2,080 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dropout_1 (Dropout) │ (None, 32) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_2 (Dense) │ (None, 1) │ 33 │ └──────────────────────────────────────┴─────────────────────────────┴─────────────────┘
Total params: 2,881 (11.25 KB)
Trainable params: 2,881 (11.25 KB)
Non-trainable params: 0 (0.00 B)
optimizer = tf.keras.optimizers.Adam(learning_rate=0.005) # defining Adam as the optimizer to be used
model_2.compile(optimizer=optimizer, loss='binary_crossentropy', metrics=['accuracy', tf.keras.metrics.Recall(name='recall')])
#batch_size = X_train.shape[0]
batch_size = 64
epochs = 40
start = time.time()
history = model_2.fit(X_train, y_train, validation_data=(X_val,y_val) , batch_size=batch_size, epochs=epochs)
end=time.time()
Epoch 1/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 2s 6ms/step - accuracy: 0.7741 - loss: 0.5302 - recall: 0.0336 - val_accuracy: 0.7970 - val_loss: 0.4546 - val_recall: 0.0049 Epoch 2/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8043 - loss: 0.4548 - recall: 0.0663 - val_accuracy: 0.8220 - val_loss: 0.4307 - val_recall: 0.2285 Epoch 3/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 4ms/step - accuracy: 0.8141 - loss: 0.4365 - recall: 0.1840 - val_accuracy: 0.8280 - val_loss: 0.4155 - val_recall: 0.2408 Epoch 4/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8203 - loss: 0.4178 - recall: 0.2533 - val_accuracy: 0.8385 - val_loss: 0.3989 - val_recall: 0.3243 Epoch 5/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8296 - loss: 0.4131 - recall: 0.2859 - val_accuracy: 0.8420 - val_loss: 0.3826 - val_recall: 0.3219 Epoch 6/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8377 - loss: 0.3972 - recall: 0.3312 - val_accuracy: 0.8495 - val_loss: 0.3688 - val_recall: 0.3563 Epoch 7/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8403 - loss: 0.3893 - recall: 0.3632 - val_accuracy: 0.8510 - val_loss: 0.3697 - val_recall: 0.3833 Epoch 8/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8476 - loss: 0.3737 - recall: 0.3608 - val_accuracy: 0.8530 - val_loss: 0.3660 - val_recall: 0.4005 Epoch 9/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8444 - loss: 0.3754 - recall: 0.3626 - val_accuracy: 0.8515 - val_loss: 0.3645 - val_recall: 0.3612 Epoch 10/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8548 - loss: 0.3675 - recall: 0.3818 - val_accuracy: 0.8485 - val_loss: 0.3627 - val_recall: 0.3956 Epoch 11/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8538 - loss: 0.3707 - recall: 0.3966 - val_accuracy: 0.8480 - val_loss: 0.3652 - val_recall: 0.4373 Epoch 12/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8515 - loss: 0.3659 - recall: 0.4077 - val_accuracy: 0.8510 - val_loss: 0.3619 - val_recall: 0.3710 Epoch 13/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8504 - loss: 0.3623 - recall: 0.3863 - val_accuracy: 0.8490 - val_loss: 0.3582 - val_recall: 0.3710 Epoch 14/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8516 - loss: 0.3636 - recall: 0.3820 - val_accuracy: 0.8505 - val_loss: 0.3608 - val_recall: 0.3882 Epoch 15/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8504 - loss: 0.3606 - recall: 0.3785 - val_accuracy: 0.8520 - val_loss: 0.3606 - val_recall: 0.3931 Epoch 16/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8539 - loss: 0.3617 - recall: 0.3904 - val_accuracy: 0.8470 - val_loss: 0.3611 - val_recall: 0.3464 Epoch 17/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8504 - loss: 0.3659 - recall: 0.3604 - val_accuracy: 0.8490 - val_loss: 0.3588 - val_recall: 0.3784 Epoch 18/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8501 - loss: 0.3607 - recall: 0.3784 - val_accuracy: 0.8500 - val_loss: 0.3584 - val_recall: 0.3857 Epoch 19/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.8461 - loss: 0.3595 - recall: 0.3834 - val_accuracy: 0.8495 - val_loss: 0.3585 - val_recall: 0.3907 Epoch 20/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8553 - loss: 0.3633 - recall: 0.3904 - val_accuracy: 0.8500 - val_loss: 0.3580 - val_recall: 0.4029 Epoch 21/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 6ms/step - accuracy: 0.8479 - loss: 0.3622 - recall: 0.3676 - val_accuracy: 0.8485 - val_loss: 0.3592 - val_recall: 0.3489 Epoch 22/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 6ms/step - accuracy: 0.8532 - loss: 0.3598 - recall: 0.3902 - val_accuracy: 0.8530 - val_loss: 0.3600 - val_recall: 0.3857 Epoch 23/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 6ms/step - accuracy: 0.8544 - loss: 0.3556 - recall: 0.4016 - val_accuracy: 0.8510 - val_loss: 0.3559 - val_recall: 0.3784 Epoch 24/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8524 - loss: 0.3577 - recall: 0.3978 - val_accuracy: 0.8500 - val_loss: 0.3614 - val_recall: 0.3489 Epoch 25/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 5ms/step - accuracy: 0.8563 - loss: 0.3489 - recall: 0.3972 - val_accuracy: 0.8525 - val_loss: 0.3579 - val_recall: 0.3538 Epoch 26/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8479 - loss: 0.3583 - recall: 0.3778 - val_accuracy: 0.8525 - val_loss: 0.3575 - val_recall: 0.3808 Epoch 27/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8534 - loss: 0.3586 - recall: 0.3929 - val_accuracy: 0.8515 - val_loss: 0.3584 - val_recall: 0.4177 Epoch 28/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8566 - loss: 0.3535 - recall: 0.4347 - val_accuracy: 0.8525 - val_loss: 0.3606 - val_recall: 0.4128 Epoch 29/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8532 - loss: 0.3576 - recall: 0.4226 - val_accuracy: 0.8510 - val_loss: 0.3570 - val_recall: 0.4029 Epoch 30/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8530 - loss: 0.3552 - recall: 0.4385 - val_accuracy: 0.8510 - val_loss: 0.3585 - val_recall: 0.3735 Epoch 31/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8544 - loss: 0.3521 - recall: 0.3942 - val_accuracy: 0.8560 - val_loss: 0.3545 - val_recall: 0.4423 Epoch 32/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8553 - loss: 0.3539 - recall: 0.4226 - val_accuracy: 0.8540 - val_loss: 0.3567 - val_recall: 0.4103 Epoch 33/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8557 - loss: 0.3502 - recall: 0.4299 - val_accuracy: 0.8525 - val_loss: 0.3605 - val_recall: 0.4079 Epoch 34/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8564 - loss: 0.3561 - recall: 0.4146 - val_accuracy: 0.8530 - val_loss: 0.3626 - val_recall: 0.4324 Epoch 35/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8565 - loss: 0.3507 - recall: 0.4456 - val_accuracy: 0.8530 - val_loss: 0.3572 - val_recall: 0.4029 Epoch 36/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 4ms/step - accuracy: 0.8565 - loss: 0.3546 - recall: 0.4068 - val_accuracy: 0.8580 - val_loss: 0.3541 - val_recall: 0.4152 Epoch 37/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8550 - loss: 0.3485 - recall: 0.4025 - val_accuracy: 0.8570 - val_loss: 0.3562 - val_recall: 0.4373 Epoch 38/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8580 - loss: 0.3464 - recall: 0.4223 - val_accuracy: 0.8530 - val_loss: 0.3573 - val_recall: 0.3931 Epoch 39/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8563 - loss: 0.3510 - recall: 0.4194 - val_accuracy: 0.8475 - val_loss: 0.3650 - val_recall: 0.3514 Epoch 40/40 94/94 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8556 - loss: 0.3553 - recall: 0.4090 - val_accuracy: 0.8510 - val_loss: 0.3564 - val_recall: 0.4177
print("Time taken in seconds ",end-start)
Time taken in seconds 20.626485586166382
plot(history,'loss')
plot(history,'accuracy')
plot(history,'recall')
results.loc[2] = ['NN with Adam/Dropout',2,'[64,32]','relu',epochs,batch_size,history.history["loss"][-1],history.history["val_loss"][-1],history.history["accuracy"][-1],history.history["val_accuracy"][-1],history.history["recall"][-1],history.history["val_recall"][-1],round(end-start,2)]
results = results.sort_index()
results
| model label | # hidden layers | # neurons - hidden layer | activation function - hidden layer | # epochs | batch size | train loss | validation loss | train accuracy | validation accuracy | train recall | validation recall | time (secs) | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | NN with SGD | 2 | [64,32] | relu | 100 | 64 | 0.33 | 0.35 | 0.86 | 0.86 | 0.47 | 0.43 | 42.23 |
| 1 | NN with Adam | 2 | [64,32] | relu | 40 | 64 | 0.31 | 0.37 | 0.87 | 0.85 | 0.51 | 0.45 | 20.51 |
| 2 | NN with Adam/Dropout | 2 | [64,32] | relu | 40 | 64 | 0.35 | 0.36 | 0.86 | 0.85 | 0.43 | 0.42 | 20.63 |
Commentary: This model reduced the already modest overfitting from the previous two models, however, the recall scores are worse, so all-in-all, not an improvement with this one.
#Applying SMOTE to handle class imbalance
X_train, y_train = smote.fit_resample(X_train, y_train)
# Print the updated shape of the training data
print("Training set shape:", X_train.shape, y_train.shape)
Training set shape: (9554, 11) (9554,)
#Show the updated balance of y_train
y_train.value_counts()
| count | |
|---|---|
| Exited | |
| 0 | 4777 |
| 1 | 4777 |
# clears the current Keras session, resetting all layers and models previously created, freeing up memory and resources.
tf.keras.backend.clear_session()
#Initializing the neural network
model_3 = Sequential()
model_3.add(Dense(64, activation="relu",input_dim = X_train.shape[1]))
model_3.add(Dense(32,activation="relu"))
model_3.add(Dense(1,activation = "sigmoid"))
#Printing the summary.
model_3.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩ │ dense (Dense) │ (None, 64) │ 768 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_1 (Dense) │ (None, 32) │ 2,080 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_2 (Dense) │ (None, 1) │ 33 │ └──────────────────────────────────────┴─────────────────────────────┴─────────────────┘
Total params: 2,881 (11.25 KB)
Trainable params: 2,881 (11.25 KB)
Non-trainable params: 0 (0.00 B)
optimizer = keras.optimizers.SGD(learning_rate=0.005, momentum=0.9) # defining SGD as the optimizer to be used
model_3.compile(optimizer=optimizer, loss='binary_crossentropy', metrics=['accuracy', tf.keras.metrics.Recall(name='recall')])
#batch_size = X_train.shape[0]
batch_size = 64
epochs = 100
start = time.time()
history = model_3.fit(X_train, y_train, validation_data=(X_val,y_val) , batch_size=batch_size, epochs=epochs)
end=time.time()
Epoch 1/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 2s 4ms/step - accuracy: 0.5156 - loss: 0.6937 - recall: 0.6299 - val_accuracy: 0.5530 - val_loss: 0.6862 - val_recall: 0.7224 Epoch 2/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.6465 - loss: 0.6476 - recall: 0.6828 - val_accuracy: 0.5800 - val_loss: 0.6772 - val_recall: 0.7027 Epoch 3/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.6635 - loss: 0.6237 - recall: 0.6784 - val_accuracy: 0.5965 - val_loss: 0.6660 - val_recall: 0.7052 Epoch 4/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.6698 - loss: 0.6098 - recall: 0.6803 - val_accuracy: 0.6180 - val_loss: 0.6510 - val_recall: 0.7076 Epoch 5/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.6854 - loss: 0.5975 - recall: 0.6926 - val_accuracy: 0.6460 - val_loss: 0.6277 - val_recall: 0.7150 Epoch 6/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7020 - loss: 0.5859 - recall: 0.7083 - val_accuracy: 0.6790 - val_loss: 0.6034 - val_recall: 0.6929 Epoch 7/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7062 - loss: 0.5760 - recall: 0.7112 - val_accuracy: 0.6940 - val_loss: 0.5919 - val_recall: 0.7027 Epoch 8/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7124 - loss: 0.5687 - recall: 0.7135 - val_accuracy: 0.7000 - val_loss: 0.5862 - val_recall: 0.7076 Epoch 9/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.7144 - loss: 0.5632 - recall: 0.7114 - val_accuracy: 0.7060 - val_loss: 0.5833 - val_recall: 0.7174 Epoch 10/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7190 - loss: 0.5583 - recall: 0.7123 - val_accuracy: 0.7100 - val_loss: 0.5816 - val_recall: 0.7224 Epoch 11/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7245 - loss: 0.5536 - recall: 0.7193 - val_accuracy: 0.7145 - val_loss: 0.5804 - val_recall: 0.7273 Epoch 12/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7289 - loss: 0.5486 - recall: 0.7221 - val_accuracy: 0.7175 - val_loss: 0.5776 - val_recall: 0.7322 Epoch 13/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7351 - loss: 0.5427 - recall: 0.7332 - val_accuracy: 0.7175 - val_loss: 0.5743 - val_recall: 0.7322 Epoch 14/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7405 - loss: 0.5356 - recall: 0.7379 - val_accuracy: 0.7245 - val_loss: 0.5708 - val_recall: 0.7469 Epoch 15/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7480 - loss: 0.5268 - recall: 0.7428 - val_accuracy: 0.7245 - val_loss: 0.5732 - val_recall: 0.7666 Epoch 16/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7545 - loss: 0.5174 - recall: 0.7504 - val_accuracy: 0.7115 - val_loss: 0.5829 - val_recall: 0.7838 Epoch 17/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 4ms/step - accuracy: 0.7577 - loss: 0.5088 - recall: 0.7571 - val_accuracy: 0.7045 - val_loss: 0.5907 - val_recall: 0.7936 Epoch 18/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7647 - loss: 0.5003 - recall: 0.7642 - val_accuracy: 0.6880 - val_loss: 0.6013 - val_recall: 0.8059 Epoch 19/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 4ms/step - accuracy: 0.7647 - loss: 0.4933 - recall: 0.7646 - val_accuracy: 0.6750 - val_loss: 0.6121 - val_recall: 0.8182 Epoch 20/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 4ms/step - accuracy: 0.7671 - loss: 0.4875 - recall: 0.7641 - val_accuracy: 0.6685 - val_loss: 0.6222 - val_recall: 0.8378 Epoch 21/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7706 - loss: 0.4827 - recall: 0.7690 - val_accuracy: 0.6630 - val_loss: 0.6284 - val_recall: 0.8452 Epoch 22/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7750 - loss: 0.4786 - recall: 0.7744 - val_accuracy: 0.6620 - val_loss: 0.6282 - val_recall: 0.8501 Epoch 23/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7741 - loss: 0.4748 - recall: 0.7697 - val_accuracy: 0.6635 - val_loss: 0.6335 - val_recall: 0.8550 Epoch 24/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7757 - loss: 0.4715 - recall: 0.7684 - val_accuracy: 0.6600 - val_loss: 0.6364 - val_recall: 0.8526 Epoch 25/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7764 - loss: 0.4690 - recall: 0.7680 - val_accuracy: 0.6570 - val_loss: 0.6409 - val_recall: 0.8673 Epoch 26/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.7752 - loss: 0.4672 - recall: 0.7663 - val_accuracy: 0.6530 - val_loss: 0.6463 - val_recall: 0.8722 Epoch 27/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.7753 - loss: 0.4657 - recall: 0.7650 - val_accuracy: 0.6485 - val_loss: 0.6488 - val_recall: 0.8747 Epoch 28/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7783 - loss: 0.4639 - recall: 0.7670 - val_accuracy: 0.6465 - val_loss: 0.6502 - val_recall: 0.8771 Epoch 29/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7794 - loss: 0.4623 - recall: 0.7692 - val_accuracy: 0.6510 - val_loss: 0.6423 - val_recall: 0.8722 Epoch 30/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7805 - loss: 0.4608 - recall: 0.7695 - val_accuracy: 0.6535 - val_loss: 0.6378 - val_recall: 0.8722 Epoch 31/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7807 - loss: 0.4594 - recall: 0.7683 - val_accuracy: 0.6550 - val_loss: 0.6362 - val_recall: 0.8747 Epoch 32/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7810 - loss: 0.4581 - recall: 0.7686 - val_accuracy: 0.6530 - val_loss: 0.6364 - val_recall: 0.8771 Epoch 33/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7837 - loss: 0.4571 - recall: 0.7724 - val_accuracy: 0.6540 - val_loss: 0.6361 - val_recall: 0.8722 Epoch 34/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7854 - loss: 0.4555 - recall: 0.7730 - val_accuracy: 0.6605 - val_loss: 0.6297 - val_recall: 0.8649 Epoch 35/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7856 - loss: 0.4553 - recall: 0.7728 - val_accuracy: 0.6615 - val_loss: 0.6245 - val_recall: 0.8649 Epoch 36/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7866 - loss: 0.4537 - recall: 0.7737 - val_accuracy: 0.6650 - val_loss: 0.6229 - val_recall: 0.8600 Epoch 37/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7863 - loss: 0.4530 - recall: 0.7727 - val_accuracy: 0.6665 - val_loss: 0.6233 - val_recall: 0.8600 Epoch 38/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 4ms/step - accuracy: 0.7867 - loss: 0.4523 - recall: 0.7738 - val_accuracy: 0.6695 - val_loss: 0.6146 - val_recall: 0.8550 Epoch 39/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 4ms/step - accuracy: 0.7879 - loss: 0.4511 - recall: 0.7740 - val_accuracy: 0.6660 - val_loss: 0.6223 - val_recall: 0.8649 Epoch 40/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 4ms/step - accuracy: 0.7884 - loss: 0.4509 - recall: 0.7749 - val_accuracy: 0.6660 - val_loss: 0.6200 - val_recall: 0.8575 Epoch 41/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7889 - loss: 0.4501 - recall: 0.7748 - val_accuracy: 0.6685 - val_loss: 0.6180 - val_recall: 0.8600 Epoch 42/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7900 - loss: 0.4490 - recall: 0.7758 - val_accuracy: 0.6675 - val_loss: 0.6205 - val_recall: 0.8624 Epoch 43/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7891 - loss: 0.4492 - recall: 0.7773 - val_accuracy: 0.6675 - val_loss: 0.6213 - val_recall: 0.8575 Epoch 44/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7890 - loss: 0.4476 - recall: 0.7777 - val_accuracy: 0.6700 - val_loss: 0.6169 - val_recall: 0.8550 Epoch 45/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7899 - loss: 0.4473 - recall: 0.7798 - val_accuracy: 0.6720 - val_loss: 0.6153 - val_recall: 0.8575 Epoch 46/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7911 - loss: 0.4463 - recall: 0.7793 - val_accuracy: 0.6730 - val_loss: 0.6170 - val_recall: 0.8575 Epoch 47/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7921 - loss: 0.4454 - recall: 0.7806 - val_accuracy: 0.6810 - val_loss: 0.6038 - val_recall: 0.8501 Epoch 48/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.7937 - loss: 0.4436 - recall: 0.7817 - val_accuracy: 0.6825 - val_loss: 0.5997 - val_recall: 0.8452 Epoch 49/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7934 - loss: 0.4434 - recall: 0.7803 - val_accuracy: 0.6810 - val_loss: 0.6037 - val_recall: 0.8452 Epoch 50/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7920 - loss: 0.4432 - recall: 0.7795 - val_accuracy: 0.6805 - val_loss: 0.6032 - val_recall: 0.8452 Epoch 51/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7940 - loss: 0.4421 - recall: 0.7806 - val_accuracy: 0.6855 - val_loss: 0.5988 - val_recall: 0.8428 Epoch 52/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7935 - loss: 0.4412 - recall: 0.7796 - val_accuracy: 0.6830 - val_loss: 0.6002 - val_recall: 0.8403 Epoch 53/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7939 - loss: 0.4406 - recall: 0.7813 - val_accuracy: 0.6855 - val_loss: 0.5993 - val_recall: 0.8403 Epoch 54/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7951 - loss: 0.4395 - recall: 0.7824 - val_accuracy: 0.6830 - val_loss: 0.6013 - val_recall: 0.8403 Epoch 55/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7961 - loss: 0.4389 - recall: 0.7844 - val_accuracy: 0.6820 - val_loss: 0.6032 - val_recall: 0.8452 Epoch 56/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7963 - loss: 0.4384 - recall: 0.7845 - val_accuracy: 0.6820 - val_loss: 0.6031 - val_recall: 0.8378 Epoch 57/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7973 - loss: 0.4380 - recall: 0.7849 - val_accuracy: 0.6875 - val_loss: 0.5954 - val_recall: 0.8354 Epoch 58/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7983 - loss: 0.4366 - recall: 0.7865 - val_accuracy: 0.6860 - val_loss: 0.5956 - val_recall: 0.8305 Epoch 59/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 4ms/step - accuracy: 0.7974 - loss: 0.4365 - recall: 0.7861 - val_accuracy: 0.6860 - val_loss: 0.5938 - val_recall: 0.8305 Epoch 60/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 4ms/step - accuracy: 0.8005 - loss: 0.4353 - recall: 0.7907 - val_accuracy: 0.6895 - val_loss: 0.5910 - val_recall: 0.8305 Epoch 61/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8001 - loss: 0.4346 - recall: 0.7899 - val_accuracy: 0.6870 - val_loss: 0.5945 - val_recall: 0.8329 Epoch 62/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.8014 - loss: 0.4338 - recall: 0.7913 - val_accuracy: 0.6890 - val_loss: 0.5931 - val_recall: 0.8305 Epoch 63/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.8014 - loss: 0.4331 - recall: 0.7918 - val_accuracy: 0.6880 - val_loss: 0.5916 - val_recall: 0.8256 Epoch 64/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8007 - loss: 0.4330 - recall: 0.7903 - val_accuracy: 0.6860 - val_loss: 0.6037 - val_recall: 0.8403 Epoch 65/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.8021 - loss: 0.4323 - recall: 0.7931 - val_accuracy: 0.6880 - val_loss: 0.5997 - val_recall: 0.8403 Epoch 66/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.8022 - loss: 0.4315 - recall: 0.7926 - val_accuracy: 0.6855 - val_loss: 0.6061 - val_recall: 0.8452 Epoch 67/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.8029 - loss: 0.4315 - recall: 0.7935 - val_accuracy: 0.6865 - val_loss: 0.6048 - val_recall: 0.8428 Epoch 68/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8023 - loss: 0.4306 - recall: 0.7931 - val_accuracy: 0.6870 - val_loss: 0.6026 - val_recall: 0.8403 Epoch 69/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.8035 - loss: 0.4294 - recall: 0.7938 - val_accuracy: 0.6940 - val_loss: 0.5890 - val_recall: 0.8280 Epoch 70/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.8028 - loss: 0.4294 - recall: 0.7927 - val_accuracy: 0.6850 - val_loss: 0.6072 - val_recall: 0.8428 Epoch 71/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8029 - loss: 0.4288 - recall: 0.7948 - val_accuracy: 0.6940 - val_loss: 0.5909 - val_recall: 0.8329 Epoch 72/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.8047 - loss: 0.4275 - recall: 0.7948 - val_accuracy: 0.6950 - val_loss: 0.5904 - val_recall: 0.8329 Epoch 73/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8045 - loss: 0.4272 - recall: 0.7946 - val_accuracy: 0.6985 - val_loss: 0.5860 - val_recall: 0.8305 Epoch 74/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.8054 - loss: 0.4266 - recall: 0.7952 - val_accuracy: 0.6990 - val_loss: 0.5833 - val_recall: 0.8280 Epoch 75/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.8065 - loss: 0.4264 - recall: 0.7968 - val_accuracy: 0.6875 - val_loss: 0.6035 - val_recall: 0.8378 Epoch 76/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8038 - loss: 0.4266 - recall: 0.7955 - val_accuracy: 0.6855 - val_loss: 0.6072 - val_recall: 0.8403 Epoch 77/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 4ms/step - accuracy: 0.8051 - loss: 0.4259 - recall: 0.7979 - val_accuracy: 0.6835 - val_loss: 0.6103 - val_recall: 0.8428 Epoch 78/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8056 - loss: 0.4254 - recall: 0.7970 - val_accuracy: 0.6835 - val_loss: 0.6092 - val_recall: 0.8428 Epoch 79/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 4ms/step - accuracy: 0.8053 - loss: 0.4250 - recall: 0.7975 - val_accuracy: 0.6825 - val_loss: 0.6082 - val_recall: 0.8403 Epoch 80/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8064 - loss: 0.4236 - recall: 0.8002 - val_accuracy: 0.6895 - val_loss: 0.6008 - val_recall: 0.8354 Epoch 81/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.8076 - loss: 0.4229 - recall: 0.8010 - val_accuracy: 0.6840 - val_loss: 0.6048 - val_recall: 0.8378 Epoch 82/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8064 - loss: 0.4221 - recall: 0.8003 - val_accuracy: 0.6850 - val_loss: 0.6052 - val_recall: 0.8378 Epoch 83/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.8081 - loss: 0.4216 - recall: 0.8025 - val_accuracy: 0.6890 - val_loss: 0.6043 - val_recall: 0.8403 Epoch 84/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.8085 - loss: 0.4214 - recall: 0.8016 - val_accuracy: 0.6845 - val_loss: 0.6068 - val_recall: 0.8403 Epoch 85/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.8089 - loss: 0.4207 - recall: 0.8022 - val_accuracy: 0.6825 - val_loss: 0.6134 - val_recall: 0.8452 Epoch 86/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.8082 - loss: 0.4196 - recall: 0.8029 - val_accuracy: 0.6850 - val_loss: 0.6113 - val_recall: 0.8452 Epoch 87/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8093 - loss: 0.4191 - recall: 0.8023 - val_accuracy: 0.6835 - val_loss: 0.6152 - val_recall: 0.8477 Epoch 88/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8091 - loss: 0.4182 - recall: 0.8035 - val_accuracy: 0.6840 - val_loss: 0.6166 - val_recall: 0.8477 Epoch 89/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8095 - loss: 0.4171 - recall: 0.8041 - val_accuracy: 0.6865 - val_loss: 0.6119 - val_recall: 0.8501 Epoch 90/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.8089 - loss: 0.4167 - recall: 0.8021 - val_accuracy: 0.6865 - val_loss: 0.6116 - val_recall: 0.8501 Epoch 91/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8100 - loss: 0.4165 - recall: 0.8038 - val_accuracy: 0.6875 - val_loss: 0.6122 - val_recall: 0.8452 Epoch 92/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8088 - loss: 0.4148 - recall: 0.8039 - val_accuracy: 0.6930 - val_loss: 0.6040 - val_recall: 0.8403 Epoch 93/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.8103 - loss: 0.4137 - recall: 0.8046 - val_accuracy: 0.6940 - val_loss: 0.6058 - val_recall: 0.8428 Epoch 94/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.8115 - loss: 0.4134 - recall: 0.8048 - val_accuracy: 0.6915 - val_loss: 0.6087 - val_recall: 0.8428 Epoch 95/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8111 - loss: 0.4124 - recall: 0.8050 - val_accuracy: 0.6925 - val_loss: 0.6059 - val_recall: 0.8403 Epoch 96/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8110 - loss: 0.4124 - recall: 0.8058 - val_accuracy: 0.6980 - val_loss: 0.5975 - val_recall: 0.8378 Epoch 97/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.8126 - loss: 0.4113 - recall: 0.8065 - val_accuracy: 0.6925 - val_loss: 0.6073 - val_recall: 0.8403 Epoch 98/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 4ms/step - accuracy: 0.8126 - loss: 0.4106 - recall: 0.8067 - val_accuracy: 0.6920 - val_loss: 0.6094 - val_recall: 0.8428 Epoch 99/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 4ms/step - accuracy: 0.8140 - loss: 0.4103 - recall: 0.8095 - val_accuracy: 0.6925 - val_loss: 0.6094 - val_recall: 0.8428 Epoch 100/100 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 4ms/step - accuracy: 0.8132 - loss: 0.4094 - recall: 0.8090 - val_accuracy: 0.6970 - val_loss: 0.6040 - val_recall: 0.8403
print("Time taken in seconds ",end-start)
Time taken in seconds 68.57739901542664
plot(history,'loss')
plot(history,'accuracy')
plot(history,'recall')
#print(history.history.keys())
results.loc[3] = ['NN SMOTE with SGD',2,'[64,32]','relu',epochs,batch_size,history.history["loss"][-1],history.history["val_loss"][-1],history.history["accuracy"][-1],history.history["val_accuracy"][-1],history.history["recall"][-1],history.history["val_recall"][-1],round(end-start,2)]
results = results.sort_index()
results
| model label | # hidden layers | # neurons - hidden layer | activation function - hidden layer | # epochs | batch size | train loss | validation loss | train accuracy | validation accuracy | train recall | validation recall | time (secs) | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | NN with SGD | 2 | [64,32] | relu | 100 | 64 | 0.33 | 0.35 | 0.86 | 0.86 | 0.47 | 0.43 | 42.23 |
| 1 | NN with Adam | 2 | [64,32] | relu | 40 | 64 | 0.31 | 0.37 | 0.87 | 0.85 | 0.51 | 0.45 | 20.51 |
| 2 | NN with Adam/Dropout | 2 | [64,32] | relu | 40 | 64 | 0.35 | 0.36 | 0.86 | 0.85 | 0.43 | 0.42 | 20.63 |
| 3 | NN SMOTE with SGD | 2 | [64,32] | relu | 100 | 64 | 0.40 | 0.60 | 0.82 | 0.70 | 0.82 | 0.84 | 68.58 |
Commentary: This model has achieved much better recall, but the diverging behavior of accuracy for train and validation suggests significant overfitting.
# clears the current Keras session, resetting all layers and models previously created, freeing up memory and resources.
tf.keras.backend.clear_session()
#Initializing the neural network
model_4 = Sequential()
model_4.add(Dense(64, activation="relu",input_dim = X_train.shape[1]))
model_4.add(Dense(32,activation="relu"))
model_4.add(Dense(1,activation = "sigmoid"))
#Printing the summary.
model_4.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩ │ dense (Dense) │ (None, 64) │ 768 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_1 (Dense) │ (None, 32) │ 2,080 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_2 (Dense) │ (None, 1) │ 33 │ └──────────────────────────────────────┴─────────────────────────────┴─────────────────┘
Total params: 2,881 (11.25 KB)
Trainable params: 2,881 (11.25 KB)
Non-trainable params: 0 (0.00 B)
optimizer = tf.keras.optimizers.Adam(learning_rate=0.005) # defining Adam as the optimizer to be used
model_4.compile(optimizer=optimizer, loss='binary_crossentropy', metrics=['accuracy', tf.keras.metrics.Recall(name='recall')])
#batch_size = X_train.shape[0]
batch_size = 64
epochs = 40
start = time.time()
history = model_4.fit(X_train, y_train, validation_data=(X_val,y_val) , batch_size=batch_size, epochs=epochs)
end=time.time()
Epoch 1/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.6459 - loss: 0.6217 - recall: 0.6060 - val_accuracy: 0.6305 - val_loss: 0.6795 - val_recall: 0.8501 Epoch 2/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7486 - loss: 0.5154 - recall: 0.7435 - val_accuracy: 0.6580 - val_loss: 0.6305 - val_recall: 0.8477 Epoch 3/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7703 - loss: 0.4804 - recall: 0.7630 - val_accuracy: 0.6625 - val_loss: 0.6034 - val_recall: 0.8354 Epoch 4/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7767 - loss: 0.4649 - recall: 0.7681 - val_accuracy: 0.6725 - val_loss: 0.5920 - val_recall: 0.8403 Epoch 5/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7817 - loss: 0.4542 - recall: 0.7775 - val_accuracy: 0.6870 - val_loss: 0.5781 - val_recall: 0.8182 Epoch 6/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7907 - loss: 0.4416 - recall: 0.7859 - val_accuracy: 0.6970 - val_loss: 0.5704 - val_recall: 0.8034 Epoch 7/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7950 - loss: 0.4349 - recall: 0.7946 - val_accuracy: 0.7125 - val_loss: 0.5507 - val_recall: 0.7912 Epoch 8/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.7979 - loss: 0.4303 - recall: 0.7980 - val_accuracy: 0.7185 - val_loss: 0.5526 - val_recall: 0.7862 Epoch 9/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7987 - loss: 0.4261 - recall: 0.7990 - val_accuracy: 0.7215 - val_loss: 0.5517 - val_recall: 0.7789 Epoch 10/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8042 - loss: 0.4217 - recall: 0.8048 - val_accuracy: 0.7245 - val_loss: 0.5524 - val_recall: 0.7862 Epoch 11/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8045 - loss: 0.4170 - recall: 0.8038 - val_accuracy: 0.7280 - val_loss: 0.5481 - val_recall: 0.7862 Epoch 12/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.8072 - loss: 0.4129 - recall: 0.8079 - val_accuracy: 0.7285 - val_loss: 0.5467 - val_recall: 0.7715 Epoch 13/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 4ms/step - accuracy: 0.8060 - loss: 0.4088 - recall: 0.8110 - val_accuracy: 0.7370 - val_loss: 0.5351 - val_recall: 0.7690 Epoch 14/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8086 - loss: 0.4032 - recall: 0.8154 - val_accuracy: 0.7585 - val_loss: 0.5071 - val_recall: 0.7273 Epoch 15/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8125 - loss: 0.3990 - recall: 0.8190 - val_accuracy: 0.7580 - val_loss: 0.5134 - val_recall: 0.7543 Epoch 16/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 4ms/step - accuracy: 0.8139 - loss: 0.3957 - recall: 0.8218 - val_accuracy: 0.7435 - val_loss: 0.5393 - val_recall: 0.7789 Epoch 17/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8151 - loss: 0.3928 - recall: 0.8249 - val_accuracy: 0.7585 - val_loss: 0.5199 - val_recall: 0.7543 Epoch 18/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 2ms/step - accuracy: 0.8200 - loss: 0.3888 - recall: 0.8320 - val_accuracy: 0.7435 - val_loss: 0.5543 - val_recall: 0.7838 Epoch 19/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8210 - loss: 0.3865 - recall: 0.8340 - val_accuracy: 0.7490 - val_loss: 0.5410 - val_recall: 0.7617 Epoch 20/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8213 - loss: 0.3833 - recall: 0.8359 - val_accuracy: 0.7520 - val_loss: 0.5399 - val_recall: 0.7666 Epoch 21/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.8211 - loss: 0.3787 - recall: 0.8334 - val_accuracy: 0.7450 - val_loss: 0.5454 - val_recall: 0.7469 Epoch 22/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8251 - loss: 0.3777 - recall: 0.8399 - val_accuracy: 0.7615 - val_loss: 0.5235 - val_recall: 0.7224 Epoch 23/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.8271 - loss: 0.3736 - recall: 0.8408 - val_accuracy: 0.7725 - val_loss: 0.5104 - val_recall: 0.7101 Epoch 24/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8303 - loss: 0.3686 - recall: 0.8404 - val_accuracy: 0.7530 - val_loss: 0.5420 - val_recall: 0.7420 Epoch 25/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8317 - loss: 0.3646 - recall: 0.8470 - val_accuracy: 0.7665 - val_loss: 0.5238 - val_recall: 0.7248 Epoch 26/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 4ms/step - accuracy: 0.8345 - loss: 0.3614 - recall: 0.8493 - val_accuracy: 0.7645 - val_loss: 0.5296 - val_recall: 0.7199 Epoch 27/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 4ms/step - accuracy: 0.8341 - loss: 0.3613 - recall: 0.8470 - val_accuracy: 0.7665 - val_loss: 0.5282 - val_recall: 0.7199 Epoch 28/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8403 - loss: 0.3566 - recall: 0.8568 - val_accuracy: 0.7660 - val_loss: 0.5293 - val_recall: 0.7150 Epoch 29/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 2ms/step - accuracy: 0.8389 - loss: 0.3533 - recall: 0.8557 - val_accuracy: 0.7555 - val_loss: 0.5445 - val_recall: 0.7346 Epoch 30/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8390 - loss: 0.3519 - recall: 0.8568 - val_accuracy: 0.7595 - val_loss: 0.5468 - val_recall: 0.7346 Epoch 31/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8417 - loss: 0.3501 - recall: 0.8579 - val_accuracy: 0.7615 - val_loss: 0.5425 - val_recall: 0.7371 Epoch 32/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8419 - loss: 0.3470 - recall: 0.8623 - val_accuracy: 0.7630 - val_loss: 0.5478 - val_recall: 0.7396 Epoch 33/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 4ms/step - accuracy: 0.8441 - loss: 0.3475 - recall: 0.8651 - val_accuracy: 0.7585 - val_loss: 0.5490 - val_recall: 0.7469 Epoch 34/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 4ms/step - accuracy: 0.8431 - loss: 0.3425 - recall: 0.8627 - val_accuracy: 0.7635 - val_loss: 0.5434 - val_recall: 0.7322 Epoch 35/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 2s 6ms/step - accuracy: 0.8490 - loss: 0.3396 - recall: 0.8671 - val_accuracy: 0.7535 - val_loss: 0.5625 - val_recall: 0.7469 Epoch 36/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8479 - loss: 0.3386 - recall: 0.8686 - val_accuracy: 0.7645 - val_loss: 0.5476 - val_recall: 0.7518 Epoch 37/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8505 - loss: 0.3344 - recall: 0.8705 - val_accuracy: 0.7670 - val_loss: 0.5402 - val_recall: 0.7371 Epoch 38/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8511 - loss: 0.3308 - recall: 0.8726 - val_accuracy: 0.7755 - val_loss: 0.5329 - val_recall: 0.7322 Epoch 39/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8519 - loss: 0.3297 - recall: 0.8754 - val_accuracy: 0.7675 - val_loss: 0.5456 - val_recall: 0.7445 Epoch 40/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8525 - loss: 0.3261 - recall: 0.8765 - val_accuracy: 0.7640 - val_loss: 0.5571 - val_recall: 0.7420
print("Time taken in seconds ",end-start)
Time taken in seconds 29.962656497955322
plot(history,'loss')
plot(history,'accuracy')
plot(history,'recall')
results.loc[4] = ['NN SMOTE with Adam',2,'[64,32]','relu',epochs,batch_size,history.history["loss"][-1],history.history["val_loss"][-1],history.history["accuracy"][-1],history.history["val_accuracy"][-1],history.history["recall"][-1],history.history["val_recall"][-1],round(end-start,2)]
results = results.sort_index()
results
| model label | # hidden layers | # neurons - hidden layer | activation function - hidden layer | # epochs | batch size | train loss | validation loss | train accuracy | validation accuracy | train recall | validation recall | time (secs) | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | NN with SGD | 2 | [64,32] | relu | 100 | 64 | 0.33 | 0.35 | 0.86 | 0.86 | 0.47 | 0.43 | 42.23 |
| 1 | NN with Adam | 2 | [64,32] | relu | 40 | 64 | 0.31 | 0.37 | 0.87 | 0.85 | 0.51 | 0.45 | 20.51 |
| 2 | NN with Adam/Dropout | 2 | [64,32] | relu | 40 | 64 | 0.35 | 0.36 | 0.86 | 0.85 | 0.43 | 0.42 | 20.63 |
| 3 | NN SMOTE with SGD | 2 | [64,32] | relu | 100 | 64 | 0.40 | 0.60 | 0.82 | 0.70 | 0.82 | 0.84 | 68.58 |
| 4 | NN SMOTE with Adam | 2 | [64,32] | relu | 40 | 64 | 0.32 | 0.56 | 0.86 | 0.76 | 0.88 | 0.74 | 29.96 |
Commentary: This model has very high recall for Train, but the Validation Metrics are much worse indicating significant overfitting.
# clears the current Keras session, resetting all layers and models previously created, freeing up memory and resources.
tf.keras.backend.clear_session()
#Initializing the neural network
model_5 = Sequential()
model_5.add(Dense(64, activation="relu",input_dim = X_train.shape[1])) #Input Layer
model_5.add(Dropout(0.3)) #Dropout Layer
model_5.add(Dense(32,activation="relu")) #Hidden Layer
model_5.add(Dropout(0.3)) #Dropout Layer
model_5.add(Dense(1,activation = "sigmoid")) #Output Layer
#Printing the summary.
model_5.summary()
Model: "sequential"
┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┓ ┃ Layer (type) ┃ Output Shape ┃ Param # ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━┩ │ dense (Dense) │ (None, 64) │ 768 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dropout (Dropout) │ (None, 64) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_1 (Dense) │ (None, 32) │ 2,080 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dropout_1 (Dropout) │ (None, 32) │ 0 │ ├──────────────────────────────────────┼─────────────────────────────┼─────────────────┤ │ dense_2 (Dense) │ (None, 1) │ 33 │ └──────────────────────────────────────┴─────────────────────────────┴─────────────────┘
Total params: 2,881 (11.25 KB)
Trainable params: 2,881 (11.25 KB)
Non-trainable params: 0 (0.00 B)
optimizer = tf.keras.optimizers.Adam(learning_rate=0.005) # defining Adam as the optimizer to be used
model_5.compile(optimizer=optimizer, loss='binary_crossentropy', metrics=['accuracy', tf.keras.metrics.Recall(name='recall')])
batch_size = 64
epochs = 40
start = time.time()
history = model_5.fit(X_train, y_train, validation_data=(X_val,y_val) , batch_size=batch_size, epochs=epochs)
end=time.time()
Epoch 1/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 2s 5ms/step - accuracy: 0.6053 - loss: 0.6538 - recall: 0.6135 - val_accuracy: 0.6040 - val_loss: 0.6819 - val_recall: 0.8378 Epoch 2/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7055 - loss: 0.5701 - recall: 0.7161 - val_accuracy: 0.6085 - val_loss: 0.6689 - val_recall: 0.8870 Epoch 3/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7414 - loss: 0.5237 - recall: 0.7434 - val_accuracy: 0.7030 - val_loss: 0.5422 - val_recall: 0.8133 Epoch 4/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7571 - loss: 0.5053 - recall: 0.7583 - val_accuracy: 0.6955 - val_loss: 0.5485 - val_recall: 0.8256 Epoch 5/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7654 - loss: 0.4868 - recall: 0.7723 - val_accuracy: 0.7110 - val_loss: 0.5499 - val_recall: 0.8305 Epoch 6/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7658 - loss: 0.4834 - recall: 0.7653 - val_accuracy: 0.7045 - val_loss: 0.5381 - val_recall: 0.8354 Epoch 7/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7719 - loss: 0.4787 - recall: 0.7772 - val_accuracy: 0.7280 - val_loss: 0.5118 - val_recall: 0.7862 Epoch 8/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7755 - loss: 0.4688 - recall: 0.7757 - val_accuracy: 0.6945 - val_loss: 0.5403 - val_recall: 0.8378 Epoch 9/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7740 - loss: 0.4730 - recall: 0.7926 - val_accuracy: 0.7265 - val_loss: 0.5138 - val_recall: 0.8034 Epoch 10/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7728 - loss: 0.4688 - recall: 0.7706 - val_accuracy: 0.7440 - val_loss: 0.4987 - val_recall: 0.7666 Epoch 11/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7858 - loss: 0.4571 - recall: 0.7921 - val_accuracy: 0.7420 - val_loss: 0.5067 - val_recall: 0.7568 Epoch 12/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7835 - loss: 0.4579 - recall: 0.7870 - val_accuracy: 0.7505 - val_loss: 0.4939 - val_recall: 0.7715 Epoch 13/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7862 - loss: 0.4554 - recall: 0.7910 - val_accuracy: 0.7485 - val_loss: 0.4977 - val_recall: 0.7543 Epoch 14/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7823 - loss: 0.4595 - recall: 0.7861 - val_accuracy: 0.7685 - val_loss: 0.4756 - val_recall: 0.7346 Epoch 15/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7894 - loss: 0.4470 - recall: 0.7883 - val_accuracy: 0.7735 - val_loss: 0.4682 - val_recall: 0.7297 Epoch 16/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7880 - loss: 0.4554 - recall: 0.7914 - val_accuracy: 0.7520 - val_loss: 0.4939 - val_recall: 0.7592 Epoch 17/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7861 - loss: 0.4491 - recall: 0.7905 - val_accuracy: 0.7515 - val_loss: 0.4899 - val_recall: 0.7641 Epoch 18/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7952 - loss: 0.4448 - recall: 0.8047 - val_accuracy: 0.7605 - val_loss: 0.4858 - val_recall: 0.7396 Epoch 19/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7947 - loss: 0.4398 - recall: 0.7934 - val_accuracy: 0.7565 - val_loss: 0.4778 - val_recall: 0.7592 Epoch 20/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7917 - loss: 0.4439 - recall: 0.7949 - val_accuracy: 0.7220 - val_loss: 0.5261 - val_recall: 0.7838 Epoch 21/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7925 - loss: 0.4473 - recall: 0.8039 - val_accuracy: 0.7680 - val_loss: 0.4620 - val_recall: 0.7322 Epoch 22/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7955 - loss: 0.4401 - recall: 0.7980 - val_accuracy: 0.7285 - val_loss: 0.5053 - val_recall: 0.7789 Epoch 23/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7921 - loss: 0.4388 - recall: 0.8005 - val_accuracy: 0.7505 - val_loss: 0.4946 - val_recall: 0.7690 Epoch 24/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7965 - loss: 0.4406 - recall: 0.8072 - val_accuracy: 0.7655 - val_loss: 0.4745 - val_recall: 0.7543 Epoch 25/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.7933 - loss: 0.4367 - recall: 0.7979 - val_accuracy: 0.7650 - val_loss: 0.4739 - val_recall: 0.7518 Epoch 26/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8005 - loss: 0.4259 - recall: 0.8074 - val_accuracy: 0.7590 - val_loss: 0.4688 - val_recall: 0.7297 Epoch 27/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 4ms/step - accuracy: 0.7982 - loss: 0.4330 - recall: 0.8033 - val_accuracy: 0.7560 - val_loss: 0.4882 - val_recall: 0.7666 Epoch 28/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7938 - loss: 0.4315 - recall: 0.7994 - val_accuracy: 0.7815 - val_loss: 0.4559 - val_recall: 0.7224 Epoch 29/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.7949 - loss: 0.4349 - recall: 0.7833 - val_accuracy: 0.7800 - val_loss: 0.4652 - val_recall: 0.7322 Epoch 30/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 4ms/step - accuracy: 0.8002 - loss: 0.4259 - recall: 0.7950 - val_accuracy: 0.7615 - val_loss: 0.4818 - val_recall: 0.7445 Epoch 31/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 5ms/step - accuracy: 0.8020 - loss: 0.4299 - recall: 0.8072 - val_accuracy: 0.7710 - val_loss: 0.4701 - val_recall: 0.7568 Epoch 32/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8026 - loss: 0.4310 - recall: 0.8084 - val_accuracy: 0.7865 - val_loss: 0.4545 - val_recall: 0.7174 Epoch 33/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8059 - loss: 0.4262 - recall: 0.8004 - val_accuracy: 0.7735 - val_loss: 0.4548 - val_recall: 0.7248 Epoch 34/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8025 - loss: 0.4240 - recall: 0.8105 - val_accuracy: 0.7820 - val_loss: 0.4650 - val_recall: 0.7273 Epoch 35/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8002 - loss: 0.4261 - recall: 0.8000 - val_accuracy: 0.7950 - val_loss: 0.4391 - val_recall: 0.7052 Epoch 36/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8078 - loss: 0.4227 - recall: 0.7995 - val_accuracy: 0.7740 - val_loss: 0.4668 - val_recall: 0.7322 Epoch 37/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.7999 - loss: 0.4266 - recall: 0.8071 - val_accuracy: 0.7865 - val_loss: 0.4514 - val_recall: 0.7125 Epoch 38/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 1s 3ms/step - accuracy: 0.8001 - loss: 0.4254 - recall: 0.8020 - val_accuracy: 0.7720 - val_loss: 0.4701 - val_recall: 0.7346 Epoch 39/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8053 - loss: 0.4255 - recall: 0.8074 - val_accuracy: 0.7825 - val_loss: 0.4549 - val_recall: 0.7125 Epoch 40/40 150/150 ━━━━━━━━━━━━━━━━━━━━ 0s 3ms/step - accuracy: 0.8115 - loss: 0.4202 - recall: 0.8111 - val_accuracy: 0.7850 - val_loss: 0.4453 - val_recall: 0.7027
print("Time taken in seconds ",end-start)
Time taken in seconds 27.55156636238098
plot(history,'loss')
plot(history,'accuracy')
plot(history,'recall')
results.loc[5] = ['NN SMOTE with Adam/Dropout',2,'[64,32]','relu',epochs,batch_size,history.history["loss"][-1],history.history["val_loss"][-1],history.history["accuracy"][-1],history.history["val_accuracy"][-1],history.history["recall"][-1],history.history["val_recall"][-1],round(end-start,2)]
results = results.sort_index()
results
| model label | # hidden layers | # neurons - hidden layer | activation function - hidden layer | # epochs | batch size | train loss | validation loss | train accuracy | validation accuracy | train recall | validation recall | time (secs) | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | NN with SGD | 2 | [64,32] | relu | 100 | 64 | 0.33 | 0.35 | 0.86 | 0.86 | 0.47 | 0.43 | 42.23 |
| 1 | NN with Adam | 2 | [64,32] | relu | 40 | 64 | 0.31 | 0.37 | 0.87 | 0.85 | 0.51 | 0.45 | 20.51 |
| 2 | NN with Adam/Dropout | 2 | [64,32] | relu | 40 | 64 | 0.35 | 0.36 | 0.86 | 0.85 | 0.43 | 0.42 | 20.63 |
| 3 | NN SMOTE with SGD | 2 | [64,32] | relu | 100 | 64 | 0.40 | 0.60 | 0.82 | 0.70 | 0.82 | 0.84 | 68.58 |
| 4 | NN SMOTE with Adam | 2 | [64,32] | relu | 40 | 64 | 0.32 | 0.56 | 0.86 | 0.76 | 0.88 | 0.74 | 29.96 |
| 5 | NN SMOTE with Adam/Dropout | 2 | [64,32] | relu | 40 | 64 | 0.42 | 0.45 | 0.81 | 0.79 | 0.81 | 0.70 | 27.55 |
Commentary: This model provides the best balance of recall and not having excessive overfitting behavior.
Model 5, NN SMOTE with Adam/Dropout, offers the best performance in terms of providing an excellent recall score and also avoiding significant overfitting. Let's look at how this model performs on the test data.
test_loss, test_accuracy, test_recall = model_5.evaluate(X_test, y_test, verbose=0)
# Printing Evaluation Results
print(f"Test Loss: {test_loss:.2f}")
print(f"Test Accuracy: {test_accuracy:.2f}")
print(f"Test Recall: {test_recall:.2f}")
Test Loss: 0.45 Test Accuracy: 0.79 Test Recall: 0.74
Print the confusion matrices for train, validation, and test.
plot_confusion_matrix(model_5, X_train, y_train)
plot_confusion_matrix(model_5, X_val, y_val)
plot_confusion_matrix(model_5, X_test, y_test)
The selected model has a Test Accuracy of 79% indicating a decent ability to correctly predict the customer outcomes. Test Recall is 74%, so while a decent score, further work could be done to reduce the number of False Negatives (customers who actually churned, but were predicted not to churn). Unfortunately the Neural Network model behaves largely as black box, so it does not provide us insights directly as to the feature importances.
From the EDA, some observations:
Power Ahead